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Geeks and Artboys 



By William Gibson 



I can, these years latee, quite clearly see the shape of the chip I had on my 
shoulder when I wrote and/or assembled the piece that now stands as my 
contribution to the dis-course charted in this collection. 

I was an artboy, or had become one in my own eyes, and I felt myself, in that 
era of unevenly emergent cyberstuff, and quite painfully, to be lumped jn with 
geeks of the first water. 

I had not yet been given to recai I that I had started out very much a geek myself, 
or seen that in so many ways I still was one. And I had been out,through much of 
the late eight-ies, on what Bruce Sterling dubbed the Virtual Chicken Circuit. 
Rather than the traditional rubber chicken, the alpha geeks of Virtual Reality and 
I had been dining on all sorts of delicacies (the bill invariably picked up by some 
branch of the local bureaucracy charged with trying to figure out the shape of 
the future). We were jnvited to dine in Tokyo, Barcelona, Venice, and Linz, 
because this was jn the heyday of the oid gloves-and-goggles paradigm of VR, 
and there was a certain buzz to the effect that this stuff might be "the next 
television." At these events, I seem to recall, there was relatively little discussion of 
the Internet, or what its jhipad: might be.The buzz was mainly around the goggies 
and the gloves, these end-user interface devices. As futurists, I'm afraid, we 
were scarcely worth the fancy dinners. As futurists so often do, we threw the 
baby out and brain-stormed the marvelous potential of the bath water. 
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The baby, an evolving and determinedly prehensile multimedia/ went right on 
evolving, and still does, no sign whatever of stopping, in a world made more 
than a littie unrecognizable by the passage of a mere fifteen years. 

By the time I could be induced, by whatever sort of editorial suasion, to sit 
down and arrange words in a row about what I thought at some particular mo- 
ment about all of this,1 had grown desperately tired of being mistaken for some- 
one who knew what he was talking about. 

Rather, I felt jt imperative that I noiknow what I was talking about. I wanted to 
be recognized for some subrational, basically shamanistic function that I was 
convinced I served. I wanted jt recognized that I was in fact an artboy. Which 
meant, at least jn my understanding of what it took to qualify, that I did not, must 
not, know what I was doing. If that which writes noveis, as someone once told me 
Henry James had said, is not that which attends dinner parties, I wanted it 
recognized that that which envisions "cyberspace" is not that which glazes over 
at the press conference following a federally funded VR fest. 

Consequently I donned the attitudinal equivalent of a black raincoat, took the 
intertextual machete to a piece of my own fiction (originally commissioned by a 
Japanese cosmetics firm), and declared my literary descent from William S. 
Burroughs. I waxed nonlinear, burned brightly at both ends, and hoped to be de- 
finitively identified thereby as an artboy. 

Had the volume you now hold existed then, I would probably have not been 
able to do that. Had I read this material, my eyes would have been opened in a 
very different way. I would have had to come to accept that Ted Nelson and 
William Burroughs, strange to think, were actually aiming, at various times, in 
the same direction. 

I had had some inkling of this upon first discovering the cut-and-paste uni- 
verse of word processing: I had seen that one could now do things, at least po- 
tentially, that Inspector Lee had scarcely dreamed of; henceforth all text was 
infinitely plastic, no scissors required. (Only recently have I had the opportunity 
to examine facsimile pages of Burroughs's cut-up "journals," which are aston- 
ishing prefigurations, in paper, of pure hypertext.) 

I would have been presented with ampie evidence that geeks and artboys have 
considerable in common, under the skin, and have had, with regard to 
multimedia, from the very start. (The very start being entirely debatable, though I 
my-self have elsewhere described cave paintings as "the first screen.") 
I had not read Douglas Engelbart or Vannevar Bush then, although I had read 
quite a lot of what is offered here on the artboy side. I did not then know, and I 
was struck by this repeatedly as I recently read or reread the texts collected. 
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"ere, that the rants of artboys tend to date quite early on, whereas the rants of 
geeks retain a certain timelessness and an enviable jntensity.The reason for this, I 
think, is that artboys of whatever era are required to expend energy in the as- 
sumption of attitude, jn the donning of that black raincoat, whereas geeks (often 
quite scarily) are not. 

I would have discovered Morton Heilig's Rube Goldberg, peculiarly Popular 
Mechantes visions of VR, neither geek nor artboy but somehow Hollywood Mas- 
terTechie (and perhaps Hollywood was where the two impulses first fused, 
cinema having been the brilliant bastard offspring of a union once unthinkable to 
anyone but a frothing Italian futurist). 

I would have discovered, to my great delight, the figure of Billy Kluver, the 
perfect recombinant sport along the supposed great divide of the Two Cultures: 
a geek (Bell Laboratories laser systems engineer) whose inner artboy (collabo- 
rationswith Rauschenberg, Johns, and Warhol) allowed himto unhesitatingly and 
smilingly describe the great New York power failure of 1966 as "a Happening." 

I would have seen a certain impulse toward nonlinearity, which I had been 
culturally conditioned to think of as the hallmark of the century's avant-garde, 
manifesting, often crucial ly, in the theorizing of Ivan Sutherland and Tim 
8erners-L.ee. 

I would, I hope, have been rightly humbled. 

Who can read Vannevar Bush's "As We May Think" today and not be amazed 
by the weird acuity of this man's imagination? These visions of hyper-efficient, 
cigar-smoking technocrats, with walnut-sized "dry" cameras strapped to their 
foreheads, of glass-topped (and walnut-sided?) desks wherein are rear-projected 
images collected "in the field,"are, in their way, so genuinely, so embarrassingly, 
so rawly, prescient as to make a science fiction writer squirm. (There was a 
reason, you see, that Roosevelt was advised by Vannevar Bush, and not by 
Robert Heinlein or Isaac Asimov. Heinlein and Asimov were, relatively speaking, 
artboys. Bush, who personally put together what we think of today as the military 
industrial complex, was a geek.) 

Today, in the opening years of a new century, I suspect that the distinction 
between the geek and the artboy means less than it ever has. Indeed, I wonder how 
effectively one can now be either to the exclusion of the other. And, indeed, I some- 
times wonder how deep a distinction jt ever was. H ippies, someone must somewhere 
have pointed out, didn't discover LSD. And of al I those who partook jn that 
particular version of the lotus, it now looks as though the ones who then shambled 
into the garage to continue fiddling with what eventual ly became the personal 
com-puter were the ones who most utterly and drastically and truly changed 
things. 
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My own early work is haunted by the collage constructions of Joseph Cornell, 
a prototypical artboy/geek (to the extent, it seems, of a sort of voluntary autism 
who fits nowhere at all jn the standard art histories, and by certain luxuriantly 
retrograde impulses in the work of Jorge Luis Borges, a poet who carne to re- 
gard his later role of librarian as inherently the more romantic of the two call- 
ings. Cornell treated the island of Manhattan as an intensely private archive of 
images, and was said to possess in photographic memory the exact contents 
of every secondhand shop in New York City, shelf by dusty shelf. Borges regarded 
the universe jtself as a library, infinitely recombinant, infinitely recursive, jn which 
a single text might exist jn variorum editions beyond number. I would add both 
these artists to the company collected here, and many more as well. 

Multimedia, in my view, is not an invention but an ongoing discovery of how 
the mind and the universes it imagines (or vice versa, depending) fit together and 
interact. Multimedia is where we have always been going. Geeks and artboys, 
emerging together from the caves of Altamira, have long been about this great 
work.This book js one start toward a different sort of history, a history cognizant 
of an impulse that seems to me always to have been with us. 

I recommend this book to you with an earnestness that I have seldom felt for 
any collection of historie texts.This js, jn large part, where the bodies (or, rather, 
the bones Of the ancestors) are buried. Assembled this way, in such provocativa 
proximity, these visions give off strange sparks.Think of it, if you like, as a cut- 
up in Burroughs's best sense, an interleaving of histories intended to open Inter- 
textual doors, some of which, given the right reader, have never before been 
opened. Perhaps you are that reader. 

If not, keep jt handy: you may be that reader one day, be you geek or artboy 
(of either gender, please) or (more likely) some evolved hybrid of the two. 

Someone, it seems, always has to be. 
— Vancouver, August 2000 
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By Randall Packerand, Ken Jordan 



kan Sutherland and Sketchpad. Courtesy of Lincoln Laboratory, MIT. 

"And so the arts are encroaching one upon another, and from a proper use 
ofthls encroachment will rise the art that istruly monumental. " 

-WASSILY KANDINSKY 



Multimedia is emerging as the defining medium of the twenty-first century. The 
World Wide Web, CD-ROMs, virtual reality arcade games, and Interactive 
installations only hint at the forms of multimedia to come. Yet the concept of 
integrated, interactive media has its own long history, an evolution that spans 
more than 150 years. Remarkably, this has been a largely untold story. 
Discussions of the develop-ment of the personal computer and the Internet 
tend to focus on a few highly successful entrepreneurs, neglecting 
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the less-known work of the engineers and artists who first sought to craft a 
medium that would appeal to all the senses simultaneously — a medium that 
would mimic and enhance the creative capacities of the human mind. Mere, then, 
is a "secret history" of multimedia: a narrative that includes the pioneering ac- 
tivities of a diverse group of artists, scientists, poets, musicians, and theorists 
from Richard Wagner to Ivan Sutherland, from Vannevar Bush to Bill Viola. 

Beginning with Wagner, subsequent generations of artists sought, and found, 
integrated forms and interdisciplinary strategies to express their concern with 
individual and social consciousness and extreme states of subjective experience. 
In the years since World War II, scientists have pursued personal computing 
and human-computer interactivity as vehicles for transforming consciousness, 
ex-tending memory, increasing knowledge, amplifying the intellect, and 
enhancing creativity.The idealistic and ideological aspirations of both groups 
have resulted in a new medium that emphasizes individual choice, free 
association, and personal expression. 

Multimedia: From Wagner to Virtual Reality brings together the major 
writings by multimedia's pioneers for the first time jn order to foster a greater 
understand-ing of its precedents, landmarks, aesthetic roots, social impact, and 
revolution-ary potential. 

Douglos Engelbart. Gourtesy ofDouglas Engelbart, Bootstrap Institute. 

Newtechnology has always been used to make media. George Lucas shot his 
latest Star Wars epic with digital cameras, though the audience experience was no 



Birth of a New Medium 
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ent than if it had been shot on celluloid. But while not all computer-based is 
multimedia, today's multimedia starts with the computer, and takes the st 
advantage of the computer's capability for personal expression. Digital 
computers were initially designed as calculating machines.The first /' electronic 
computer, the EI\IIAC,was built by the United States military dur- World War II to 
produce ballistics tables for artillery jn battle. Computers «en were clumsy, 
hulking devices — the ENIAC had eighteen thousand vacuum Oes, and 
measured fifty feet by thirty feet — that did calculations for scientific esearch. 
Only a handful of scientists considered the possibility of personal com-rting for 
creative purposes by nonspecialists. 

The first scientist to think seriously of this potential was Vannevar Bush. In s 
1945 article "As We May Think," he outlined "a future device for individual jse, 
which is a sort of mechanized private file and library." Before the ENIAC was 
:;mpleted, Bush was already contemplating how information technology could 
chance the individual'scapabiiity for creative thought. "The human mind ...op- 
erates by association," Bush observed. The device that he proposed, which he '- 
amed the memex, enabled the associative indexing of information, so that the -. 
;der's trail of association would be saved inside the machine, available for ref- 
erence at a later date. This prefigured the notion of the hyperlink. While Bush 
never actually built the memex, and while his description of it relied on 
technology that predated digital information storage, his ideas had a profound 
influence on the evolution of the personal computer. 

In the years immediately after the war, under the shadow of the atomic bomb, 
the scientific establishment made a concerted effort to apply recent advance- 
ments jn technology to humanitarian purposes. In this climate, Norbert Wiener 
completed his groundbreaking theory on cybernetics. While Wiener did not live 
to see the birth of the personal computer, his book, The Human Use of Human 
Beings, has become derigueurior anyone investigating the psychological and 
sociocultural implications of human-machine interaction. Wiener understood that 
the quality of our communication with machines affects the quality of our inner 
lives. His ap-proach provided the conceptual basis for human-computer 
interactivity and for our study of the social impact of electronic media. 

Bush and Wiener established a foundation on which a number of computer 
scientists associated with the Advanced Research Projects Agency (ARPA) — a 
United States government-funded program to support defense-related research 
in the 1960s — began to build. Leading ARPA's effort to promote the use of 
computers in defense was the MIT psychologist and computer scientist J.C.R. 
Licklider, author of the influential article "Man-Computer Symbiosis." Defying 
the conventional wisdom that computers would eventually rival human intelli- 
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gence rather than enhance it, Licklider 
proposed that the computer be developed 
as a creative collaborator, a tool that could 
extend human intellectual capabil-ity and 
improve a person's ability to work 
efficiently. 

While at ARPA/ Licklider put significant 
resources toward the pursuit of his vision. 
Among the scientists he supported was 
Douglas Engelbart, who since the mid-1950s 
had been seeking support for the 
development of a digital informa-tion 
retrieval system inspired by Bush's memex. 
ARPA funding enabled Engelbart to 
assemble a team of computer scientists and 
psychologists at the Stanford Research 
Institute to create a "tool kifthat would, as 
he phrased it, "augment human intellect." 
Dubbed theoNLine System (NLS), itspublic 
debut in 1968 at the Fall Joint Computer 
Conference in San Francisco was a 
landmark event in the history of computing. 
Engelbart unveiled the NLS before a room 
of three thousand computer scientists, who 
sat jn rapt attention for nearly two hours 
while he demonstrated some of his major 
innovations, including the mouse, win-dows 
for text editing, and electronic mail. 
Engelbart was making jt possible, for the 
first time, to reach virtually through a 
computer's interface to manipulate in- 
formation. Each of his innovations was a 
key step toward an interface that al-lowed 
for intuitive interactivity by a nonspecialist. At 
the end of his presentation, he received a 
standing ovation. 

However,thecontributionsof the NLS 
went beyond innovation regarding the 
computer interface. Engelbart and his 
colleagues also proposed that creativity 
could be enhanced by the sharing of ideas 



and Information through computers used as 
Communications devices. The oNLine 
System had its computers wired into a local 
network, which enabled them to be used for 
meaningful collabora-tion between co- 
workers. Engelbart understood that the 



Personal computer would not only 
augment intelligence but augment 
communication as well. In 1969 his 
research in on-line networking 
carne to fruition with the creation of 
ARPANET, the forerunner of the 
Internet.The first message on the 
ARPANET, in fact, was sent to Engelbart's 
lab at SRI from UCLA. 

Engelbart's N LS pioneered some of the 
essential components necessary for the 
personal computer, but it would be up to a 
new generation of engineers to ad-vance 
computing so jt could embrace multimedia. 
As a graduate student jn the late 1960s, 
Alan Kay wrote a highly influential 
Ph.D. thesis proposing a personal 
Information management device that, in 
many ways, prefigured the laptop. In 1970, 
as research jn Information science was 
shifting from East Coast universi-ties and 
military institutions to private digital 
companies jn Silicon Valley, Kay was 
invited to join the new Xerox PARC (Palo 
Alto Research Center) in California. PARC's 
mandate was no less than to create "the 
architecture of infor-mation for the future." 
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At PARC, Alan Kay pursued his jdea for the Dynabook — a notebook-sized 
puter that enabled hyperlinking, was fully interactive, and integrated all 
ia.Withthe Dynabook, digital multimedia carne into being. Echoing Licklider, 
Blbart, and colleagues at PARC, Kay declared the personal computer a jn 
its own right. It was a "meta-medium," as he described it jn his 1977 ' 
"Personal Dynamic Media," capable of being "all other media." While the r - 
abook remained a prototype that was never built, the work that carne from • 
development, including the invention of the Graphical User Interface (GUI) : 
subsequent breakthroughs in dynamic computing, was incorporated into the : 
true multimedia computer, the Xerox Alto. 



ie Integration ofthe Arts 



Pepsi Pavilion. Photo by Harry Skunl Courtesy ofBilly Kluver, E.A. T. 



By proposing that the Dynabook be a "meta-medium" that unifies all media 
within a single interactive interface, Alan Kay had glimpsed into the future. But 
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he may not have realized that his proposal had roots in the theories of the 
nineteenth-century German opera composer, Richard Wagner. 

In 1849, Wagner introduced the concept of the Gesamtkunstwerk, or Total Art- 
work, in an essay called "The Artwork of the Future." It would be difficult to 
overstate the power of this idea, or its influence. Wagner's description of the 
Gesamtkunstwerk is one of the first attempts in modern art to establish a practical, 
theoretical system for the comprehensive integration of the arts. Wagner sought 
the idealized union of a//the arts through the "totalizing," or synthesizing, effect 
of music drama — the unification of music, song, dance, poetry, visual arts, and 
stagecraft. His drive to embrace the full range of human experience, and to re- 
flect jt jn his operas, led him to give equal attention to every aspect of the final 
production. He was convinced that only through this integration could he attain 
the expressive powers he desired to transform music drama into a vehicle capa- 
ble of affecting German culture. 

Twentieth-century artists have continued the effort to heighten the viewer's 
experience of art by integrating traditional ly separate disciplines into single works. 
Modern experience, many of these artists believed, could be evoked only through 
an art that contained within itself the complete range of perception. "Old- 
fashioned"forms limited to words on a page, paint on a canvas, or music from an 
instrument were considered inadequate for capturing the speed, energy, and con- 
tradictions of contemporary ufe. 

In their 1916 manifesto "The Futurist Cinema," FT Marinetti and his rev- 
olutionary cohorts declared film to be the supreme art because jt embraced all 
other art forms through the use of (then) new media technology. Only cinema, 
they claimed, had a "totalizing" effect on human consciousness. Less than a 
decade later, in his 1924 essay describing the theater of the Bauhaus, "Theater, 
Circus, Variety," Laszlo Moholy-Nagy called for a theater of abstraction that 
shifted the emphasis away from the actor and the written text, and brought to 
the fore every other aspect of the theatrical experience. Moholy-Nagy declared 
that only the synthesis of the theater's essential formal components — space, 
composition, motion, sound, movement, and light — into an organic whole could 
give expression to the full range of human experience. 

The performance work of John Cage was a significant catalyst in the contin-uing 
breakdown of traditional boundaries between artistic disciplines after World War 
II. In the late 1940s, during a residency at Black Mountain College jn North 
Carolina, Cage organized a series of events that combined his interest in 
collaborative performance with his use of indeterminacy and chance operations 
in musical composition. Together with choreographer Merce Cunningham and 
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Robert Rauschenberg and Jasper Johns, Cage devised theatrical experi-that 
furthered the dissolution of borders between the arts. He was par-rly 
attracted to aesthetic methods that opened the door to greater :ipation of 
the audience, especial ly if these methods encouraged a height-l awareness of 
subjective experience. Cage's use of indeterminacy and chance-technique shifted 
responsibility for the outcome of the work away from jartist, and weakened yet 
another traditional boundary, the divide between art- 
and audience. 

I age's work proved to be extremely influential on the generation of artists : 
carne of age in the late 1950s. Alian Kaprow, Dick Higgins, and Nam June ^k 
were among the most prominent of the artists who, inspired by Cage, de-ed 
nontraditional performance techniques that challenged accepted notions , 
categorization, and composition, leading to the emergence of genres such the 
Happening, electronic theater, performance art, and interactive jnstalla- 

Kaprow, who coined the term "Happening," was particularly interested in 
blurring the distinction between artwork and audience. The ultimate integrated 
*rt, he reasoned, would be without an audience, because every participant would 
se an integral part of the work. As he wrote jn his 1966 primer, "Untitled 
Guide-'esfor Happenings, ""The une between art and lifeshould be kept as fluid, 
and rerhaps indistinct, as possible.'This approach led to a performance style that 
pj-: A eered deliberate, aesthetically conceived group interactivity in a composed 
en-, ronment. Happenings artists devised formal elements that allowed 
participants »_ne freedom to make personal choices and collective decisions that 
would affect ~i)e performance. 

In this climate, artists became increasingly interested in integrating 
technology into their work. While technology clearly played a significant role in 
twentieth-century arts (such as photography, film, and video, as well as various 
fine arts genres), it was not until Bell Labs scientist Billy Kluver placed the po- 
tential of advanced engineering into the hands of artists in New York that 
integrated worksof art and technology began to flourish. Kluver conceived the- 
notion of equal collaboration between artist and engineer. He pioneered forms of 
art and technology that would have been unimaginable to the artist without the 
engi-neer's cooperation and creative involvement.With Robert Rauschenberg, 
Kluver created several of the earliest artworks to integrate electronic media and 
to en-courage a participatory role for the audience, including Oracle (1963-1965) 
and 

In 1966 Kluver and Rauschenberg, along with Robert Whitman and Fred 
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Waldhauer, cofounded E.A.T. (Experiments in Art and Technology) to bring 
artists and engineers together to create new works. EA.T.'s most ambitious pro- 
duction was the Pepsi Pavilion, designed for the Osaka Expo 70 in Japan — a 
tremendously ambitious collaborative, multimedia project that involved more 
than seventy-five artists and engineers. As Kluver explained, audience partici- 
pation was at the heart of their interests: "The initial concern of the artists who 
designed the Pavilion was that the quality of the experience of the visitor should 
involve choice, responsibility, freedom, and participaron.The Pavilion would not 
tell astory or guide the visitor through a didactic, authoritarian experience.The 
visitor would be encouraged as an individual to explore the environment and 
compose his own experience." 

During this period, the British artist and theorist Roy Ascott began to explore 
the use of computers in artistic expression. One of the first theoretical attempts 
to integrate the emerging fields of human-computer interactivity and cybernet- 
ics with artistic practice was Ascott's article "Behavioral Art and the Cybernetic 
Vision," from 1966-1967. Ascott noted that the computer was "the supreme tool 
that... technology has produced. Used in conjunction with synthetic materials 
it can be expected to open up paths of radical change in art." Ascott saw that 
human-computer interaction would profoundly affect aesthetics, leading artists 
to embrace collaborative and interactive modes of experience. 

When Alan Kay arrived at Xerox PARC in 1970, the foundation was jn place 
for a multimedia that synthesized al I the existing art forms, and presented them 
in an environment that allowed for meaningful interactivity. With the Dynabook, 
the interactive Gesamkunstwerk was brought into the digital realm, and put on- 
line. 

Through the Looking Glass 

The fantasy of being transported into another world, to be taken wholly jnto 
an imaginary realm, is a primal desire. With computer-based multimedia, en- 
counterswith immersive, virtual worldswill soon become commonplace. Virtual 
reality, after all, js a logical extension of the integration of the arts. It is also an 
ideal environment for applying our knowledge of human-computer interactivity. 

There is an evocative echo jn virtual environments of the earliest known form 
of human expression — the prehistoria cave paintings found at such sites as the 
caves of Lascaux in the south of France.These immersive environments, dating 
from 15,000 b.c, are thought by scholars to have been theaters for the 
performance of rituals that integrated all forms of media and engaged all the 
senses. 
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i the walls were painted animal jimages and shamanist scrawls, but the envi- 
T>nment in which the paintings appeared was surely as significant as the paint- 
themseJves. As Joseph Campbell described \t, "these magica/ spots occur far i 
the natural entrantes of the grottos, deep within the dark, wandering chill »jdors 
and vast chambers, so that before reaching them one has to experience the ful I 
force of the mystery of the cave itself." To encounter the cave paintings was to 
immerse the self in an otherworldly domain, which would heighten con- 
sciousness and trigger altered states of perception. 

There are many examples of immersion jn the history of art. The Dyonisian 
rituals of Greek theater and the great cathedrals of Europe are two obvious 
examples. Richard Wagner's Gesamtkunstwerk was driven by a vision of theater in 
which the audience loses jtself in the veracity of the drama, creating an immer- 
sive experience. As he wrote in The Artwork of the Future, "the spectator transplants 
himself upon the stage, by means of all his visual and aural faculties.'To 
facilitate his vision, Wagner reinvented the conventions of the opera house, and 
in 1876 opened the FestpielhausTheater in Bayreuth, Germany, with the first 
complete production of The Ring cycle.The Festpielhaus, with its employment of 
Greek amphitheatrical seating, surround-sound acoustics, the darkening of the 
house, and the placement of musicians in an orchestra pit, focused the 
audience's un- 



SccttFuher. VIEW Head-Mounted Display. Courtesy of Scott Fisher, NASA-Ames Research Center. 
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divided attention on the dramatic action. His intent was to maximize the 
suspension of disbeljef,to drawthe viewer into an illusionary world staged within 
the proscenium arch. 

In the 1950s, a similar vision inspired the American cinematographer Mor- 
ton Heilig to propose a "cinema of the future" that would surround the audience 
with facsimiles of life so convincing they would believe themselves to be trans- 
ported to another domain. Such a cinema, he wrote, "would faithfully reproduce 
man's outer world as perceived in his consciousness, it will eventually learn to 
create totally new sense materials for each of the senses . .. [that] they have 
never known before, and to arrange them into forms of consciousness never be- 
fore experienced by man in his contact with the outer world." 

While Heilig devised a theoretical framework that applied the technologies of 
his day toward the achievement of virtual experience, it was only as a conse- 
quence of advances in computer science that the jmmersion his work suggested 
became possible. 

By 1965, Ivan Sutherland had already achieved legendary status among 
computer scientists as the inventor of Sketchpad, the first interactive graphics 
software. In a short paper published that year, Sutherland mused over the options 
available to the engineer to display computer data, to create a "a looking glass" 
into what he described as a "mathematical wonderland." It seemed reasonable 
for him to suggest that "The Ultimate Display" (as the paper was titled) would 
represent this data in three-dimensional form, allowing the construction of en- 
tirely believable three-dimensional, computer-controlled, virtual worlds. How- 
ever, like Heilig before him, Sutherland took this suggestion one step further.'The 
ultimate display," he wrote, "would ... be a room within which the computer can 
control the existence of matter." He suggested that such a display could present 
realities heretofore only imagined, as jf seen through Aiice's looking glass. 
Sutherland's proposal was startling, but jt launched an entire field of scientific 
inquiry. 

It also fueled the imagination of a generation of artists. One of the first to 
consider the possibilities of digitally constructed virtual experiences was Myron 
Krueger. In the early 1970s, Krueger created the pioneering works Meiap/ayand 
Videoplaceto explore the potential of computer-mediated interactivity.These works 
were interactive artistic environments, influenced by Happenings, designed to give 
participants freedom of choice and opportunities for personal expression. Video- 
p/acealso connected participants in different locations through networked 
technologies, creating the illusion of shared space. As Krueger later wrote about 
the piece, "our teleconference created a place that consisted of the information 
we 
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shared ... a world in which full physical participation would be possible. j*orld 
is an 'artificial reality.' " 



Notion that information might be treated as a "place" inspired the re-i of N 
jcholas Negroponte and Richard A. Bolt at MIT's Architecture Ma-Group jn the 
late 1970s. Sutherland's proposal for a 3-D display implied a computer could 
represent information spatially. But how would such a '. actually work? To 
address this question, Negroponte and Bolt built the ary "media room/' drawing 
ideas from the heretofore divergent fields of dge management and architecture 
theory. As Bolt describes in his essay tial Data-Management/'their system 
involved "the creation of a plausible commodious 'virtual' space at the 
computer interface, together with a way g around in that space quickly and 
easily/This virtual space, which 1 referred to as "Dataland," was projected 
onto the walls of the "media n," surrounding the user, who navigated this 
data territory from a chair , pped with a joystick, touch screens, and a 
microphone, among other devices. During the 1980s, several engineering 
projects took the concept of virtual ironment display systems even further. 
Perhaps the most significant of these led by Scott Fisher at the N ASA-Ames 
Research Center. Fisher's intent was r gage the entire nervous system in a 
multisensory presentation of virtual ce — extending multimedia beyond the 
screen.The Ames VIEW system (an acronym for Virtual Interface 
Environmental Workstation) included a headset «rth two small liquid crystal 
display screens, a microphone for speech recog-»rtion, earphones for 
surround-sound effects, a head-tracking device, and a :a:aglove to recognize 
the user's gestures and place them within the virtual en-nronment. The direction 
this work pointed jn was clear. As Fisher wrote in his 1989 article "Virtual 
Interface Environments," "with full body tracking ca-pability, it would also be 
possible for users to be represented in this [virtual] space by life-size virtual 
representations of themselves jn whatever form they choose." 

Immersive environments, Fisher observed, could give birth to a new form of 
participatory, Interactive electronic theater. Among the first to pursue such a the- 
ater was the artist Jeffrey Shaw, who began to incorporate virtual reality into 
his installations in the mid-1980s. Shaw's installations are 3-D landscapes with 
a theatricality that entices the spectator to actively explore them. His work The 
Legible City, for example, presents an urban environment in which buildings are 
literal ly composed from words; the participant reads the texts by navigating 
through virtual city streets wearing a head-mounted display, riding a stationary 
bicycle that is, in fact, the work's interface. 
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By the late 1980s, the possibilities for this kind of Interactive, virtual theater had 
taken firm hold of the public's imagination. In Neuromancer, his widely read novel 
from 1984, William Gibson described in palpable detail a future in which virtual 
reality was a fact of ufe. Echoing Myron Krueger's notion that telecon-ferencing 
created a "place" that consisted of shared information, G ibson's char-acters 
inhabited a virtual environment made possible by the networking of computers, 
which he named "cyberspace." G ibson's cyberspace provided the first literary 
definition for the computers, hubs, servers, and databases that make up the 
matrices of the network. H js discussion of cyberspace was so tangible — and 
seductive, with its suggestion that any computer hacker could "jack-in to the ma- 
trix" for an encounter with a sexy avatar — it became a touchstone for every en- 
gineer, artist, and theorist working in the field. 

Marcus Novak took Gibson's description of virtual environments as the start- 
ing point for his own theoretical and artistic explorations. In his essay from 
1991, "Liquid Architecture in Cyberspace," he follows the pioneering work of 
Sutherland, Fisher, and Gibson, et al., to its logical conclusion, and notes its pro- 
found implications for architecture, our notions of space, and our attitudes to- 
ward the organization of information. He notes that in cyberspace, since all 
structure is programmable, all environments can be fluid.The artist who designs 
these jmmersive digital habitats will be able to transcend the laws of the physi- 
cal world. As a consequence, architectural forms built jn cyberspace can respond 
to the viewer, encouraging provocative and illuminating interactions. In 
cyberspace, architecture becomes a form of poetry. 

The artist Char Davies also investigates the mutable aspects of jmmersive 
environments, concentrating on the way the body responds to changes in the 
virtual space. For her jmmersive interactive multimedia work Osmose (1995), 
Davies augmented a head-mounted display and dataglove with sensors that 
allow the user to navigate imaginary worlds through breath and balance. The 
effect is to create a contemplative experience for the participant, which is 
triggered by the subtle, unconscious movements of the body. 

While most research in virtual reality aims to project the viewer into a digital 
environment by means of a head-mounted display — as in the work of such 
media artists as Jaron Lanier, Brenda Laurel, and Jenny Holzer — someengineers 
nave taken an alternative approach. In the early 1990s, Daniel Sandin and 
Thomas DeFanti conceived of a virtual reality system that places the human body 
directly inside a computer-generated environment. They describe their system, 
called the CAVE (an acronym for Cave Automatic Virtual Environment) jn their 
article"Room with a View":"Unlike users of the video-arcadetype of virtual re- 
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system, CAVE 'dwellers'do not need to 
wear helmets, which would limit their Jf 
and mobility jn the real world ... to 
experience virtual reality." Instead, its 
jn the CAVE are surrounded by an 
immersive, digital "cave paint-which 
brings the evolution of immersion ful I 
circle, back to the prehistoric )f 
Lascaux, and humankind's earliest 
efforts at personal expression. 

Integrated Datawork 

Imn Hershman, Lorna. © Lynn Hershman. Courtesy ofLynn Hershman. 

It was Vannevar Bush who, in 1945, determined the chief narrative charac- 
teristic of multimedia by proposing a mechanical device that operated literally 
as we may think.'The challenge, as he saw it, wasto create a machine that sup- 
oorted the mind's process of free association. Bush noted how jdeas tend to 
evolve in a nonlinear, idiosyncratic fashion. H is memex would be a tool that could 
supplement this aspect of human creativity by organizing its media elements to 
eflect the dynamics of the rnind at play. 

Douglas Engelbart expanded on Bush's premise. His quest to "augment 
human intellect," as he aptly phrased it, was based on the insight that the open 
flow of ideas and information between collaborators was as important to 
creativity as private free association was.The personal computer, as he 
envisioned jt, would not only allow for the arrangement of data in idiosyncratic, 
nonlinear formats, but by connecting workstations to a data-sharing network 
and turning 
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them into Communications devices, Engelbart's oNLine System allowed for a 
qualitative leap in the collaboration between individuals — almost as if colleagues 
could peer into one another's minds as part of the creative process. In the early 
1960s, experiments with networked personal computing promised the nonlinear 
organizaron of information on a grand scale. 

While few recognized this possibility at the time, it inspired a series of influ- 
ential theoretical writings by the rogue philosopher Ted Nelson. Working outside 
the academic and commercial establishments, following his own strongly held 
convictions, Nelson devised an elaborate system for the sharing of information 
across computer networks. Called Xanadu, this system would maximize a com- 
puter's creative potential. Central to Nelson's approach was the "hyperlink," a 
term he coined jn 1963, inspired by Bush's notion of the memex's associative 
trails. Hyperlinks, he proposed, could connect discrete texts in nonlinear se- 
quences. Using hyperlinks, Nelson realized, writers could create "hypertexts," 
which he described as "nonsequential writing" that let the reader make decisions 
about how the text could be read jn other than linear fashion. As he observed in 
his landmark book from 1974, Computer Lib/Dream Machines, "the structures 
of ideas are notsequential/'With hypertext, and its multimedia counterpart, 
"hypermedia," writers and artists could create works that encouraged the user to 
leap from one jdea to the next jn a series of provocative juxtapositions that 
presented alterna-tives to conventional hierarchies. 

Nelson's jnsights were paralleled by experiments in the literary avant-garde 
that challenged traditional notions of linear narrative. In his book, he refers to 
Vladimir Nabokov's Pale Fire ana Julio Cortazar's Hopscotchasiwo noveis that use 
unconventional branching structures to encourage the reader's active 
collaboration in the construction of the story. As we have already seen, 
experimental performances inspired by John Cage — including Happenings, 
interactive instal-lations, and performance art — also gave rise to a variety of 
nonlinear narrative strategies. But perhaps the most prescient explorer of this 
terrain was the nov-elist William S. Burroughs. 

Like Ted Nelson, Burroughs was deeply suspicious of established hierarchies. 
He was especially interested in writing techniques that suggest the spontaneous, 
moment-by-moment movement of the mind, and how nonlinear writing might ex- 
pand the reader's perception of reality.Through his use of the cut-up and fold-in 
techniques, which he described in his 1964 essay "The Future of the Novel," 
Burroughs treated the reading experience as one of entering into a 
multidirectional web of different voices, ideas, perceptions, and periods of time. 
He saw the cut-up as a tool that let the writer discover previously undetected 
connections be- 
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i things, with potentially enlightening and subversiva results. With the cut- 
Burroughs prefigured the essential narrative strategy of hypertext and its 
' to allow readers to leap across boundaries in time and space, unce the 
invention of the electric telegraph by Samuel Morse in the 1830s, entators have 
been noting the transformation of our concepts of space time by wired 
technology. From the telegraph to the telephone to television isatellite 
Communications, modern telecommunications has eradicated geo-lic borders, 
and made speed a central factor in modern life.This effect was anly 
acknowledged as long ago as 1868, when, at a banquet held in honor :-se's ufe 
achievement, he was toasted for having "annihilated both space Itime in the 
transmission of intelligence.The breadth of the Atlantic, with all 
es, is as nothing." 

Artists have grappled with the implications of this technology since its jn-on;the 
narrative experiments of literary authors reflect this current in mod-art. Ted 
Melson's concept of hypertext representad a profound effort to put vrchnology 
toward the service of personal, idiosyncratic expression. Nelson ame an 
evangelist for hypertext, publishing articles, speaking at conferences, eading the 
gospel wherever he could. One of those places was Brown Univer-y, which during 
the 1980s became a hotbed of literary explorations of the i. At Brown, the 
literary critic George Landow and his colleagues developed ertexttools,such as 
Intermedia, which allowed authors with little experience programming to invent 
new genres of creative writing. In his own work, iow applied a trained critical 
eye to the formal aspects of hypertext, mak-rxj connections to the 
poststructural textual analysis of critics like Roland Barthes and Jacques 
Derrida. Just as academic theoretical discourse was ques-:oning the centrality of 
the author in the production of texts, hypermedia sug-gested that, in a future of 
networked digital media, responsibility would shift from author to reader, actively 
encouraging the reader's collaboration jn shaping a -arrative. Inventive 
hypertext fictions by authors such as Michael Joyce, Mark Amerika, Shelley 
Jackson, and Stuart Woulthrop explored the literary possibil- ties of this 
nonlinear form of storytelling. 

During this period, media artists whose roots lay jn performance and video 
also began investigating hypermedia as a means of exploring new forms for 
:elljng stories. Artists such as Lynn Hershman and Bill Viola were drawn to the 
computer's ability to break down linear narrative structures. Viola approached 
the medium as a repository for evocative images that could be projected on 
screens jn installations, "with the viewer wandering through some three- 
dimensional, possibly life-sized field of prerecorded or simulated scenes evolving 



xxx Overture 



jn time/' as he described it. Hershman was among the first to create digital art- 
works using Interactive media, jn such pieces as Deep Contad, from 1989. She in- 
troduced interactivity into her work to combat the loss of intimacy and control 
brought about by the dom'mance of media such as radio and television. Her use 
of hypermedia allowed the viewer to choose directions inside the artworks com- 
plex branching structure, and shape a personal experience from it. 

By the late 1980s, multimedia, which had been at the fringe of the arts and 
sciences, reached critical mass and went mainstream. Marc Canter, who devel- 
oped the first commerdal multimedia authoring system, was a chief catalyst. 
Canter pioneered software tools that artists and designers used to create 
multimedia on their personal computers. His authoring systems synthesized text, 
images, animation, video, and sound into a single integrated work, using 
hyperlinks and other hypermedia techniques to connect its various elements. 

In 1974,Ted Nelson had declared that "The real dream is for A everything'to 
be jn the hypertext." It was a proposal that echoed Marshall McLuhan's influ- 
ential observation that "after more than a century of electric technology, we have 
extended our central nervous system itself jn a global embrace, abolishing both 
space and time as far as our planet is concerned." But despite Nelson's best ef- 
forts, and many advances in the fields of hypermedia and computer-based 
telecommunications,the global hypermedia library he envisioned remained more 
dream than reality. Most innovations jn hypermedia focused on closed systems, 
such as the CD-ROM and interactive installations, rather than on open systems 
using a computer network. 

In 1989 Tim Berners-Lee, a young British engineer working at CERN, the 
particle physics laboratory in Geneva, Switzerland, circulated a proposal for ar 
in-house on-line document sharing system, which he described modestly as "a 
N web' of notes with links." After getting a grudging go-ahead from his superiors, 
Berners-Lee dubbed this system the World Wide Web. The Web, as he designee 
it, combined the Communications language of the Internet with Nelson's hyper 
text and hypermedia, enabling links between files to extend across a global net 
work. It became possible to link every document, sound file, or graphic on th» 
Web in an infinite variety of nonlinear paths through the network. And insteat 
of being created by a single author, links could be written by anyone participa! 
ing in the system. Not only did the open nature of the Web lend itself to a wjd< 
array of interactive, multimedia experiences, but by hewing to a nonhierarchica 
structure and open protocols, Berners-Lee's invention became enormously pop 
ular, and led to an explosion jn the creation of multimedia. By 1993 the Web ha< 
truly become an international phenomenon. 
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; success of the Web seemed to confirm the intuition of artists engaging jn I 
media that in the future, a global media database would inspire new forms 
ssion. Roy Ascott, for example, had already been exploring the creative lilities 
of networking since the 1980s. He was interested in the notion of 
ace/'aterritory of information in which al I dataexists in acontinual pre-:- 
_tside the traditional definitions of time and space available for use in end-: 
.-xtapositions. Ascott considers dataspace a new type of Gesamtkunstwerk, or 
atenwerk, as he calis it, in which networked Information is integrated into 
jartwork. In such an environment, Ascott writes, "meaning is not something 
by the artist, distributed through the network, and received by the ob-. 
Meaning is the product of interaction between the observer and the system, :ent 
of which is in a state of flux, of endless change and transformation." This 
notion of the artwork as a territory for interaction, as a locus of com- j. cations 
for a community, echoes the Happenings of a previous generation, artists Kit 
Galloway and Sherrie Rabinowitz were particularly intrigued by j possibilities 
of extending collective experience through Communications tech-gies. 
Beginning jn 1975, they presided over a series of "teleperformances" used 
satellite hookups to link performers across great distances. In such arks as 
Hole jn Space, from 1980, they made the many-to-many spontaneous 
ommunication of a large number of participants the conteni of their 
"teleper-:Tnance." Inspired by the technology's promise, the couple opened 
the Electric Cafe International in Santa Monica in 1989 as a research center 
and jrformance space to explore new forms of networked, collaborative art 
be-taeen artists and communities. 

On-line role-playing games have also become laboratories for exploring this 
acsect of interactivity. As the social theorist Sherry Turkle has pointed out, on- 
'"•e communities, such as Multi-User Dungeons (MUD), "are anewgenre of col- 
aborative writing, with things jn common with performance art, street theater, 
-iprovisation theater, commedia deN'arte, and script writing." Pavel Curtis cre- 
ated one of the earliest MUDs, LambdaMOO, in 1990 at Xerox PARC.Though 
it consisted only of text, its interactive quality, made possible through intricate 



storytelling devices via the Internet, gave participants the illusion of 
immersion 

a virtual environment. Interaction in the on-line environment, Curtis claimed, 
creates a kind of social behavior which "in some ways ... is a direct mirror of 
behavior in real ufe." 

Mirroring real life, computer-based interactivity has proven to be an invita- 
tion to inspire the viewer to shape his or her own experience of reality through 
electronic mediation The theorist Janet Murray describes this as "agency," 
which 
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she defines as u the satisfying power to take meaningful action and see the results 
of our decisions and cholees." This form of agency has evolved in the network 
environment as a system for empowering the viewer to engage with the artist as 
a creative partner. 

Throughout history, art has often been referred to as a mirror of life. But by 
building upon the twin notions of association and collaboration, computer-based 
multimedia may well become more than a mirror of life. Already we have seen 
how multimedia blurs the boundaries between life and art, the personal and the 
mediated, the real and the virtual. The implications of these tendencies we are 
only now beginning to grasp. 



The Future Is Under Construction 



Marcos Novak, Voice 3 + 4 Maze Blue. © Marcos Novak. Courtesy of 
Marcos Novak.The breadth and potential of multimedia lends itself to Utopian 
proposals. The French media theorist Fierre Levy describes multimedia as 
belonging to a tra-jectory of planetary evolution that runs from DNA to 
cyberspace — an are that follows pure information as it reaches toward its 
most evolved form of expres-sion. He proposes that today's global networks 
will usher in an era of "collective intelligence," and suggests that "cyberspace 
constitutes a vast, unlimited field ... designed to interconnect and provide 
interface for the various methods of cre-ation ; recording, communication and 
simulation." His enthusiastic perspective evokes a world of intriguing 
possibilities. 
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! same time, we are al I aware of the dystopian qualities of the 24/7 in-: 
juggernaut that is being delivered across the globe through an ever | 
*phisticated telecommunications network. We read daily about the new 
[encroachment on privacy, its opportunity for abuse, and the specter of 
control that it might make possible.These dangers are real.There is i between 
opposing positions at the heart of the Internet — between those its potential for 
an open, freewheeling exchange of art and ideas and t«*K> see its 
pervasiveness as an opportunity to expand upon the marketing- broadcast 
model of twentieth-century media — and it is not at all clear 1 Utopian or 
dystopian visions will ultimately prevail. Ihe articles in this book serve as a 
poignant reminder of the intentions of dia's pioneers.Their words, typically 
written during the heat of invention, a passionate involvement with higher 
ideais. To a remarkable degree, ; scientists, artists, and theorists share a 
commitment to forms of media and jnications that are nonhierarchical, open, 
collaborative, and reflective of movement of the mind at play. It js, in sum, an 
extraordinary vision. But er we will achieve it js an unresolved question. 



Toward a DeFinition of Multimedia 

ne wake of postmodernist practice, computer-based media has resisted defini — 
and for good reason: definitions are confining. They reduce the range of po- 
trTi'al jn the object defined by drawing attention away from what lies outside 
the »a;l of definition.This is a particular concern with new media, because one 
of its =r.ractions is its fluid, multifarious character, its permeable walls. Digital 
media's peculiar nature challenges traditional categories; this jn itself is an 
aspect of its radical character. Butthere is value jn proposing and discussing 
alternative definitions of digital media — even jf these definitions are contingent, 
bracketed by cir-istances. In fact, it may be best to regard them as contingent, 
because our i<perience with digital media is so fresh, and where it leads so 
unclear. The definitions of today will inevitably be replaced tomorrow, as new 
applications for digital media emerge over time. 

Definitions are meant to establish a shared vocabulary that can focus argu- 
ment — and often, covertly, to achieve a politically motivated purpose.The pur- 
pose of this book is overt: If, as Marshall McLuhan suggests, we literally 
construct the world we inhabit through the design and deployment of our media 
technologies — because they enable certain behaviors while discouraging oth- 
ers — then the social and political ramifications of how we define and address 
the 
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emerging digital media are undeniable. By identifying a subject's key 
character-istics, we begin to say what it is and what it js not. For digital media 
this is par-ticularly critical; if the digital arts community does not lead the 
discussion about how to define digital multimedia, and the types of behaviors it 
should or shouldn't encourage, other interests, like governments and 
corporations, will force a defi-nition on us. 

The interests of for-profit entities often do not coincide with those of the cre- 
ative community. In the case of digital, the multinational media corporations 
have made clear that their intent js to maintain the legacy paradigms of 
twentieth-century media (which are hierarchical, broadcast-based, and author- 
centered) rather than support the emergence of challenging new media forms 
(which are, at their best, rhizomatic, peer-to-peer, and interactive). It js in 
their interest to force compromises from the technology that will protect their 
traditional busi-nesses — compromises that effectively gut the most democratic, 
and creatively en-gaging, aspects of digital media. If it was up to these 
powerful companies, twenty-first-century media devices would likely do no 
more than act as delivery platforms for the media formats of the last century. 

Today's situation js much different from the way new media forms have 
emerged in the past. In the days of Gutenberg, the success of movable type did 
not depend on the coordinated acceptance of printing standards across 
medieval Europe. Rather, local innovations could emerge and take hold, and 
get adopted gradually. Regional ecosystems of media practice emerged over 
time; those that best suited the needs of society spread, establishing forms for 
personal expres-sion that improved through use. 

The introduction of centralized, industrial forms of communication in the 
nineteenth century — like the telegraph, photography, telephones, audio 
recording, and cinema — required more global efforts at standardizaron. If a 
telegraph op-erator didn't know Morse code, then the telegraph became less 
valuable.The drive to make Morse code a universal standard went hand in glove 
with the expansion of the telegraph into increasingly remote regions. Still, 
significantly, each media standard of the day was unique to itself. Standards 
that emerged for photography paper had no influence on standards that were 
set for the telephone. Each medium grew independently, and so could find its 
way as a form of expression, before settling into a relatively rigid system with 
its own set of rules. 

Digital multimedia is a departure from this established model, because it 
jn-corporates traditional ly independent media forms into a single system. So 
the standards set for digital Communications will affect them all, 
simultaneously. 
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er, digital multimedia requires an unprecedented level of global coor- j wetl as 
a massive technical infrastructure and widespread user base, zs, the 
infrastructure is expensive. It demands standards agreed to by »nunity. Digital 
media calis for a far greater level of planning and de-Bource commitment 
than what we are familiar with from the past, lis reason, there is a need for 
a definition of digital media that brings No rts most radical characteristics. If 
a television network trumpets the *click-to-buy TV shopping expresses 
digital media's greatest potential, 
. ear way to say why that is not the case. 

• "-as been written about narrow aspects of the digital media experience, swr, 
little critical work has been done to show how these separate aspects »ne 
into a whole.This book is an attempt to identify the core principies that, 
bound together, articulate the inherent capabilities in digital media that ! 
toward new forms of personal expression. Our intent is to draw a une be- 
ithe mainstream media forms of the past and a possible future. Though the 
implementations of digital media are still in development (and will con-! to 
be, relentlessly, given the f reedom to do so), we identif ied basic concepts : 
oersist, regardless of the technologies being used by an artist or engineer in i 
soecific situation. Could these concepts suggest a trajectory for future 
devel-|qpnent and provide a way to measure if digital media js achieving 
what it js ca-:of? 

There are five characteristics of new media that, jn aggregate, define it as 
a »edium distinct from all others. These concepts set the scope of the form's 
ca-aaoilities for personal expression; they establish its ful I potential: 

Integration: the combining of artistic forms and technology into a 

hybrid form of expression. Interactivity: the ability of the 
user to manipulate and affect her 

experience of media directly, and to communicate with . others through 
media. Hypermedia: the linking of separate media elements to one 

another to 

create a trail of personal association. Immersion: the 
experience of entering into the simulation or 

suggestion of a three-dimensional environment. Narrativity: 
aesthetic and formal strategies that derive from the 



and medua presentaron. 

* 



xxxvi Overture 



Together, these five concepts offer a definition of digital media that pushes toward 
the technical and aesthetic frontiers of the form. 

Integration, of course, \s the foundation of multimedia; the combining of dif- 
ferent media into a single work js intrinsic to multimedia practice. While tech- 
nology has always played a role in the development of forms of expression (since 
all media are technologies in their own right), beginning in the mid-twentieth cen- 
tury there was a deliberate effort to incorporate technology as material, as a 
thing in itself, into artistic practice.This work, championed most visibly by Bell 
Labs engineer Billy Kluver, made technology an explicit aspect of the creation of 
art. This led, jn turn, to artists exploring the formal properties of electronic 
media and computers, jn order to make an art that is computer-specific. Because 
computer output can mimic traditional media, jt lends itself to artworks that blur 
the lines between media and between disciplines, just as in consciousness the dis- 
tinctions between different media forms (image, text, sound, movement) are less 
than absolute. 

Interactivity js an overused word that is jn danger of losing jts meaning. 
However, asoriginally conceived by Norbert Wiener, Douglas Engelbart,and oth- 
ers, interactivity has extraordinary promise.The term needs to be reclaimed from 
those who abuse it (by using it to describe home-shopping TV channels, for in- 
stance). By interactivity we specifically mean the ability of the user to alter 
media she comes in contact with, either alone or in collaboration with others. 
Reading a text is not an interactive experience; interactivity implies changing the 
words of the text jn some way — adding to them, reorganizing them, engaging with 
them in a way that affects their appearance on the screen. Digital media is in- 
herently dynamic, changeable. Interactivity exploits this quality, and encourages 
a creative engagement by the user that leaves jts mark on the artwork. Just as 
a conversation js a two-way experience that affects both parties, interactivity js 
an extension of our instinct to communicate, and to shape our environment 
through communication. 

Hypermedia may prove to be the most profound contribution that the 
computer has made to aesthetics. By making a persistent link between media 
objects, the user can now easily share her private path through them. Never 
before has it been so simple to make your own nonlinear method of navigating 
through ideas and information available to others. At the same time, using 
hypermedia, all traditional media forms tend to have the same weight. By writing 
links, you decide how to place emphasis on one media object jn relationship to 
another; context determines relative importance.Text leads to image leads to 
sound jn just the way the mind works. 
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But while hypermedia is potent in and of itself, without interactivity hyper-i 
would be limited to a way of browsing extant Items, rather than engaging 'with 
them. Interactivity is what empowers hypermedia, making it more :the 
experience of consciousness encountering the world. In life, one thought : to 
another, which leads you to your notebook, where you reread a line of tfien 
cross out one word and replace it with a different one. Without inter-:»« :>, 
hypermedia would place you in a state of continual passivity, frustrating f impulse 
to engage with what you encounter. 

Like hypermedia, jimmersion is a digitally enabled method for mimicking an of 
consciousness. The arts have long been concerned with accurately re-fccting 
private sensory perceptions. Tne history of each art form is replete with ements 
that claim this as their objective; similarly, integration has been led rthe desire 
to combine art forms in a way that reflects our sensual apprehen-of the world. 
Digital technology allows us to pursue this impulse even fur-through the 
creation of fully realized virtual environments. It is also true :~at, even when 
digital media does not suggest a convincing three-dimensional < '-.ual space, it 
encourages the use of spatial metaphors for the arrangement of — ormation. One 
obvious example is the Web, which lends itself to architectural ;' geographic 
methods of "navigation," rather than adhering to linear forms of o A ganization. 

The interreliance between these key characteristics culminates in the wide 
range of nonlinear narrative forms that digital media lends itself to. Our methods 
for self-expression grow out of an ongoing collaboration with the tools we use to 
give that expression a recognizable shape. Working with these tools, we find 
ways to capture nuances of personal experience so that we can share them with 
others. Before digital technology, our tools led us toward linear modes of 
expression. However, the dynamic nature of databases and telecommunications 
networks open up possibilities for alternative narrative structures that come 
closer to replicating the internal associative tendencies of the mind. Artists like 
Lynn Hershman, Roy Ascott, and Bill Viola saw this potential early on and have 
explored approaches to narrativity that make full use of integration, interactiv- 
ity, hypermedia, and jimmersion in their digital artworks.The narrative forms pi- 
oneered by these artists, and the many others who share their interests, are 
effectively blueprints for digital Communications to come in this century. 

With the Dynabook in the early 1970s, Alan Kay invented a machine that in- 
corporated al I five of these characteristics for the first time, giving birth to digital 
multimedia. However, much computer-based media today does not reflect tnese 
characteristics. A Tale of Two Cities screened on a personal digital assistant 
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them. Interactivity is what empowers hypermedia, making it more :the 
experience of consciousness encountering the world. In life, one thought : to 
another, which leads you to your notebook, where you reread a line of tfien 
cross out one word and replace it with a different one. Without inter- :•« :>, 
hypermedia would place you in a state of continual passivity, frustrating f 
impulse to engage with what you encounter. 

Like hypermedia, jimmersion is a digitally enabled method for mimicking an of 
consciousness. The arts have long been concerned with accurately re-fccting 
private sensory perceptions. Tne history of each art form is replete with 
ements that claim this as their objective; similarly, integration has been led 
rthe desire to combine art forms in a way that reflects our sensual apprehen- 
of the world. Digital technology allows us to pursue this impulse even fur- 
through the creation of fully realized virtual environments. It is also true :~at, 
even when digital media does not suggest a convincing three-dimensional < 
'-.ual space, it encourages the use of spatial metaphors for the arrangement of 
— ormation. One obvious example is the Web, which lends itself to 
architectural ;' geographic methods of "navigation," rather than adhering to 
linear forms of o A ganization. 

The interreliance between these key characteristics culminates in the wide 
range of nonlinear narrative forms that digital media lends itself to. Our 
methods for self-expression grow out of an ongoing collaboration with the tools 
we use to give that expression a recognizable shape. Working with these tools, 
we find ways to capture nuances of personal experience so that we can share 
them with others. Before digital technology, our tools led us toward linear 
modes of expression. However, the dynamic nature of databases and 
telecommunications networks open up possibilities for alternative narrative 
structures that come closer to replicating the internal associative tendencies 
of the mind. Artists like Lynn Hershman, Roy Ascott, and Bill Viola saw this 
potential early on and have explored approaches to narrativity that make full 
use of integration, interactiv-ity, hypermedia, and jmmersion in their digital 
artworks.The narrative forms pi-oneered by these artists, and the many others 
who share their interests, are effectively blueprints for digital Communications 
to come in this century. 

With the Dynabook in the early 1970s, Alan Kay invented a machine that 
in-corporated al I five of these characteristics for the first time, giving birth to 
digital multimedia. However, much computer-based media today does not 
reflect tnese characteristics. A Tale of Two Cities screened on a personal digital 
assistant 
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remains a nineteenth-century novel; not everything digital js multimedia. Yet 
with the addition of animated jilustrations, hyperlinked text, an on-line 
discussion group, or, more ambitiously, a MUD-like role-playing game that 
takes place jn a virtual Paris at the time of the Revolution, Dickens's text 
might — for good or for jll — become the basisfor a multimedia work. Justas 
movies have been based on noveis since the silent era, we are seeing 
multimedia adaptations of classic works, some of which may one day become 
classics themselves. Given the wide array of forms that multimedia currently 
takes — Internet art, virtual reality installations, graphical on-line chat spaces, 
and real-time networked performance, and many more to come — it might be 
fair to say that the medium's only defining element is its mutability, that we 
can't possibly predict the variety of its various mani-festations. Perhaps 
multimedia's most consistent quality will be jts relentlessly changing nature. 
In this book, one further attempt at definition has guided all the others: 
Multimedia is the form that makes the most complete use of the computer's 
potential for personal expression. 



Integration 

Wagner 

k Outlines oFthe Artwork oF the Future," The 
Artwork oftheFuture (1849) 

1 



Richard Wagner. 

"Whereas the public, that representaron ofdaily Ufe, forgets the 
confines ofthe auditorium, and Uves and breathes now only in 
the artwork which seems to jt as Ufe itself, and on the stage 
which seems the wide expanse ofthe whole World. " 

4 Richard Wagner 

« For Richard Wagner, the "Artwork of the 
Future" represented a rejection of lyric 
opera, which the German composer 
considered hopelessly superficial, a tired 
showcase for pompous divas. Yet the 
implications of this landmark publi-cation go 
well beyond the transformation of opera. 
Wagner believed that the fu-ture of music, 
music theater, and al I the arts lay in an 
embrace of the "collective art-work," a fusion 
of the arts that had not been attempted on 
this scale since the classic Greeks. In this 
essay, we find the first comprehensive 
treatise argu-ing for the synthesis of the arts, 



or, as Wagner defines jt, the 
Gesamtkunstwerk, the total artwork. 

Wagner was convinced that the separate 
branches of art — music, archi-tecture, 
painting, poetry, and dance — would attain 
new poetic heights when put to the service 
of the drama, which he viewed as the ideal 
medium for achiev-ing his vision. His 
totalizing approach to music theater also 
foreshadowed the experience of virtual 
reality. Scenic painting, lighting effects, and 
acoustical design were intended to render 
an entirely believable "virtual" world, jn 
which the proscenium arch serves as the 
interface to the stage environment. In 
1876, twenty-seven years after The Artwork 
of the Future, Wagner took this approach 
even further when he opened the famous 
Festpielhaus Theater in Bayreuth, Germany, 
where his theatrical innovations included 
darkening the house, surround-sound 
reverberance, the orchestra pit, and the 
revitaliza-tion of the Greek amphitheatrical 
seating to focus audience attention onto the 
stage. 

While Wagner's hyperromantic rhetoric 
may seem archaic today, there is acumen 
and foresight jn his approach to the 
Gesamtkunstwerk and the synthesis of the 
arts, which illuminates contemporary 
notions of multimedia. » 



. . . ArtlStIC Man can only fully conteni himself by uniting every branch o Art 
into the common Artwork: in every segregation of his artistic faculties he is u n free, 
not fully that which he has power to be; whereas in the common Artwork hi isfree, 
and fully that which he has power to be. 

The true endeavour of Art is therefore all-embracing: each unit who is in 
spired with a true art-instinct develops to the highest his own particular facul 
ties, not for the glory of these special faculties, but for the glory of genera 
Manhood in Art. 

The highest conjoint work of art is the Drama: it can only be at hand in a 
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xspossible fulness, when in it each separate branch ofart is at hand in its own ut- 
mut fulness. 

The true Drama is only conceivable as proceeding from a common urgence 
ofaten art towards the most direct appeal to a common public. In this Drama, 
each «parate art can only bare its utmost secret to their common public 
through a mu-mal parleying with the other arts; for the purpose of each 
separate branch ofart can only be fully attained by the reciprocal agreement 
and co-operation of all the fctanches in their common message. 
Arthitecture can set before herself no higher task than to frame for a 
fellowship >f artists, who in their own persons portray the life of Man, the 
special sur-loundings necessary for the display of the Human Artwork. Only 
that edifice is built according to Necessity, which answers most beiittingly an 
aim of man: the highest aim of man is the artistic aim; the highest artistic aim 
— the Drama. In buildings reared for daily use, the builder has only to answer 
to the lowest aim of men: beauty is therein a luxury. In buildings reared for 
luxury, he has to sat-isfy an unnecessary and unnatural need: his fashioning 
therefore is capricious, unproductive, and unlovely. On the other hand, in the 
construction of that edifice whose every part shall answer to a common and 
artistic aim alone, — thus in ihe building of the Theatre, the master-builder 
needs only to comport himself as artist, to keep a single eye upon the art- 
work. In a perfect theatrical edifice, Art's nr£a alone gives law ana measure, 
down even to the smallest detail. This need 15 twofold, that ofgiving and that 
ofreceiving, which reciprocally pervade and condition one another. The 
Scene has firstly to comply with all the conditions of -space" imposed by the 
joint (gemeinsam) dramatic action to be displayed thereon: but secondly, it 
has to fulfil those conditions in the sense of bringing this dramatic action to the 
eye and ear of the spectator in intelligible fashion. In the arrangement of the 
spacefor the spectators, the need for optic and acoustic un-derstanding of 
the artwork will give the necessary law, which can only be ob-ed by a union 
of beauty and fitness in the proportions; for the demand of the collective 
(gemeinsam) audience is the demand for the artwork, to whose com- 
prehension it must be distinctly led by everything that meets the eye. 1 Thus 
the A >ectator transplants himself upon the stage, by means of all his visual and 
aural tculties; while the performer becomes an artist only by complete 
absorption into the public. Everything, that breathes and moves upon the 
stage, thus breathes and moves alone from eloquent desire to impart, to be 
seen and heard within those walls which, however circumscribed their space, 
seem to the actor from his scenic standpoint to embrace the whole of 
humankind; whereas the public, that 
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representative of daily life, forgets the confines of the auditorium, and lives and 
breathes now only in the artwork which seems to it as Life itself, and on the stage 
which seems the wide expanse of the whole World 

Here Landscape-painting enters, summoned by a common need which she 
alone can satisfy. What the painter's expert eye has seen in Nature, what he now, 
as artist, would fain display for the artistic pleasure of the full community, he 
dovetails into the united work of all the arts, as his own abundant share. Through 
him the scene takes on complete artistic truth: his drawing, his colour, his glow- 
ing breadths of light, compel Dame Nature to serve the highest claims of Art. 
That which the landscape-painter, in his struggle to impart what he had seen and 
fathomed, had erstwhile forced into the narrow frames of panel-pictures, — what 
he had hung up on the egoist's secluded chamber-walls, or had made away to the 
inconsequent, distracting medley of a picture-barn, — therewith will he henceforth 
ful the ampie framework of the Tragic stage, calling the whole expanse of scene 
as witness to his power of re-creating Nature. The illusion which his brush and 
finest blend of colours could only hint at, could only distantly approach, he will 
here bring to its consummation by artistic practice of every known device of op- 
tics, by use of all the art of "lighting." The apparent roughness of his tools, the 
seeming grotesqueness of the method of so-called "scene-painting," will not of- 
fend him; for he will reflect that even the finest camel's-hair brush is but a hu- 
miliating instrument, when compared with the perfect Artwork; and the artist has 
no right to pride until he isfree, Le., until his artwork is completed and alive, and 
he, with all his helping tools, has been absorbed into it. But the finished artwork 
that greets him from the stage will, set within this frame and held before the 
common gaze of full publicity, immeasurably more conteni him than did his 
earlier work, accomplished with more delicate tools. He will not, forsooth, repent 
the right to use this scenic space to the benefit of such an artwork, for sake of his 
earlier disposition oia flat-laid scrap of canvas! For as, at the very worst, his work 
re-mains the same no matter what the frame from which it looks, provided only it 
bring its subject to intelligible show: so will his artwork, in this framing, at any rate 
effect a livelier impression, a greater and more universal understanding, than the 
whilom landscape picture. 

The organ for all understanding of Nature, is Man: the landscape-painter 
had not only to impart to men this understanding, but to make it for the first time 
plain to them by depicting Man in the midst of Nature. Now by setting his 
artwork in the frame of the Tragic stage, he will expand the individual man, to 
whom he would address himself, to the associate manhood of full publicity, and 
reap the satisfaction of having spread his understanding out to that, and made it 
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partner in his joy. But he cannot fully bring about this public understanding until 
he allies his work to a joint and all-intelligible aim of loftiest Art; while this aim 
luclf will be disclosed to the common understanding, past all mistaking, by the 
actual bodily man with all his warmth of life. Of all artistic things, the most di- 
rectly understandable is the Dramatic-Action (Handlung), for reason that its 
art is not complete until every helping artifice be cast behind it, as it were, and 
gen-oine life attain the faithfullest and most intelligible show. And thus each 
branch of art can only address itself to the understanding iri proportion as its 
core — whose relation to Man, or derivation from him, alone can animate and 
justify the artwork — is ripening toward the Drama. In proportion as it passes 
over into Drama, as it pulses with the Drama's light, will each domain of Art 
grow all-intelligible, completely understood and justified. 2 

On to the stage, prepared by architect and painter, now steps Artistic 
Man, as Natural Man steps on the stage of Nature. What the statuary and the 
historical painter endeavoured to limn on stone or canvas, they now limn 
upon them-iflves, their form, their body's limbs, the features of their visage, and 
raise it to the consciousness of full artistic life. The same sense that led the 
sculptor in his grasp and rendering of the human figure, now leads the Mime in 
the handling and demeanour of his actual body. The same eye which taught 
the historical painter, in drawing and in colour, in arrangement of his drapery 
and composition of his groups, to find the beautiful, the graceful and the 
characteristic, now orders the whole breadth of actual human show. Sculptor 
and painter once freed the Greek Tragedian from his cothurnus and his mask, 
upon and under which the real man could only move according to a certain 
religious convention. With justice, did this pair of plastic artists annihilate the 
last disfigurement of pure artistic man, and thus prefigure in their stone and 
canvas the tragic Actor of the Future. As they once descried him in his 
undistorted truth, they now shall let him pass into re-ality and bring his form, 
in a measure sketched by them, to bodily portrayal with all its wealth of 
movement. 

Thus the illusion of plastic art will turn to truth in Drama: the plastic artist 
will reach out hands to the dancer, to the mime, will lose himself in them, and 
thus become himself both mime and dancer. — So far as lies within his power, 
he will have to impart the inner man his feeling and his will-ing, to the eye. 
The breadth and depth of scenic space belong to him for the plastic 
message of his stature and his motion, as a single unit or in union with his 
fellows. But where his power ends, where the fulness of his will and feeling 
impels him to the uttehng of the inner man by means ofSpeech, there will the 
Word proclaim his plain and conscious purpose: lie becomes a Poet and, to be 
poet, a tone-artist (Tonkunstler). 
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But as dancer, tone-artist, and poet, he still is one and the same thing: nothing 
other than executant, artistic Man, who, in thefullest measure ofhisfaculties, im- 
parts himselfto the highest expression ofreceptive power. 

It is in him, the immediate executant, that the three sister-arts unite their 
forces in one collective operation, in which the highest faculty of each comes to 
its highest unfolding. By working in common, each one of them attains the power 
to be and do the very thing which, of her own and inmost essence, she longs to 
do and be. Hereby: that each, where her own power ends, can be absorbed 
within the other, whose power commences where her's ends, — she maintains 
her own purity and freedom, her independence as that which she is. The 
mimetic dancer is stripped of his impotence, so soon as he can sing and speak; 
the creations of Tone win all-explaining meaning through the mime, as well as 
through the poet's word, and that exacdy in degree as Tone itself is able to 
transcend into the mo-tion of the mime and the word of the poet; while the Poet 
first becomes a Man through his translation to the flesh and blood of the 
Performer: for though he metes to each artistic factor the guiding purpose 
which binds them all into a common whole, yet this purpose is first changed 
from "will" to "can" by the poet's Will descending to the actor's Can. 

Not one rich faculty of the separate arts will remain unused in the United Art- 
work of the Future; in it will each attain its first complete appraisement. Thus, 
especially, will the manifold developments of Tone, so peculiar to our 
instrumental music, unfold their utmost wealth within this Artwork; nay, Tone will 
incite the mimetic art of Dance to entirely new discoveries, and no less swell the 
breath of Poetry to unimagined fill. For Music, in her solitude, has fashioned for 
herself an organ which is capable of the highest reaches of expression. This 
organ is the Orchestra. The tone-speech of Beethoven, introduced into Drama by 
the orchestra, marks an entirely fresh departure for the dramatic artwork. While 
Architecture and, more especially, scenic Landscape-painting have power to set 
the executant dramatic Artist in the surroundings of physical Nature, and to 
dower him from the exhaustless stores of natural phenomena with an ampie and 
significant background — so in the Orchestra, that pulsing body of many-coloured 
harmony, the personating individual Man is given, for his support, a stanchless 
elemental spring, at once artistic, natural, and human. 

The Orchestra is, so to speak, the loam of endless, universal Feeling, from 
which the individual feeling of the separate actor draws power to shoot aloft to 
fullest height of growth: it, in a sense, dissolves 3 the hard immobile ground of the 
actual scene into a fluent, elastic, impressionable aether, whose unmeasured bot- 
tom is the great sea of Feeling itself. Thus the Orchestra is like the Earth from 
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i Antaeus, so soon as ever his foot had grazed it, drew new immortal life-. By 
its essence diametrically opposed to the scenic landscape which sur-s the 
actor, and therefore, as to locality, most rightly placed in the deepened id 
outside the scenic frame, it at like time forms the perfect comple-tof these 
surroundings; inasmuch as it broadens out the exhaustless physi-F dement 
of Nature to the equally exhaustless emotional element of artistic . These 
elements, thus knit together, endose the performer as with an at-f •oopheric 
ring of Art and Nature, in which, like to the heavenly bodies, he moves i in 
fullest orbit, and whence, withal, he is free to radiate on every side his j and 
his views of Ufe — broadened to infinity, and showered, as it were, on ! dbbnees 
as measureless as those on which the stars of heaven cast their rays of 



The place in which this wondrous process comes to pass, is the Theatric r: 
the collective art-work which it brings to light of day, the Drama. But to forre 
his own specific nature to the highest blossoming of its contents in this one and 
highest art-work, the separate artist, like each several art, must quell each self- 
sh. arbitrary bent toward untimely bushing into outgrowths unfurthersome to 
^»e whole; the better then to put forth all his strength for reaching of the 
highest common purpose, which cannot indeed be realised without the unit, 
nor, on the other hand, without the unit's recurrent limitation. 

This purpose of the Drama, is withal the only true artistic purpose that ever 
can be fully realised; whatsoever lies aloof from that, must necessarily lose 
itself in the sea of things indefinite, obscure, unfree. This purpose, however, 
the separate art-branch will never reach alone, 4 but only all together; and 
therefore the most universal is at like time the only real, free, the only 
universally intelligible Art-work. 

— translated by william ashton ellis 



FT. MarinelN, Bruno Corra, 
Emilio Settimelli, Arnaldo Ginna, 
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F. T. Marinetti. Photo by Coletti. 



'We shall set in motion the words- jn-freedom that smash the boundaries 
ofliterature as they march towards painting, music, noise-art, and throw a 
marvelous bridge between the word and the real object. " 
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drama (always predictable), and killing the book (always tedious and oppres- 
sive). The necessities of propaganda will force us to publish a book once in 
a while. But we prefer to express ourselves through the cinema, through 
great ta-bles of words-in-freedom and mobile illuminated signs. 

With our manifesto "The Futurist Synthetic Theatre," with the victorious 
tours of the theatre companies of Gualtiero Tumiati, Ettore Berti, Annibale 
Ninchi, Luigi Zoncada, with the two volumes oi Futurist Synthetic Theatre con- 
taining eighty theatrical syntheses, we have begun the revolution in the Italian 
prose theatre. An earlier Futurist manifesto had rehabilitated, glorified, and 
per-fected the Variety Theatre. It is logical therefore for us to carry our vivifying 
en-ergies into a new theatrical zone: the cinema. 

At first look the cinema, born only a few years ago, may seem to be 
Futurist already, lacking a past and free from traditions. Actually, by 
appearing in the guise oi theatre without words, it has inherited all the most 
traditional sweepings of the literary theatre. Consequently, everything we have 
said and done about the stage applies to the cinema. Our action is legitimate 
and necessary in so far as the cinema up to now has been and tenes to 
remain profoundly passeist, whereas we see in it the possibility of an 
eminently Futurist art and the expressive medium most adapted to the 
complex sensibility ofa Futurist artist. 

Except for interesting films of travel, hunting, wars, and so on, the film- 
makers have done no more than inflict on us the most backward-looking 
dramas, great and small. The same scenario whose brevity and variety may 
make it seem ad-vanced is, in most cases, nothing but the most trite and pious 
analysis. Therefore all the immense artistic possibilities of the cinema still rest 
entirely in the future. 

The cinema is an autonomous art. The cinema must therefore never copy 
the stage. The cinema, being essentially visual, must above all fulfill the 
evolution of painting, detach itself from reality, from photography, from the 
graceful and solemn. It must become antigraceiul, deforming, impressionistic, 
Synthetic, dy-namic, free-wording. 

one must free the cinema as an expressive medium in order to make it the 
ideal instrument ofa new art, immensely vaster and lighter than all the 
existing arts. We are convinced that only in this way can one reach 
Uiatpolyexpressiveness towards which all the most modern artistic researches 
are moving. Today the Futurist cinema creales precisely the polyexpressive 
symphony that just a year ago we announced in our manifesto "Weights, 
Measures, and Prices of Artistic Genius." The most varied elements will enter 
into the Futurist film as expressive means: from the slice of life to the streak of 
colour, from the conventional line to words-in-freedom, from chromatic and 
plastic music to the music of objects. In 
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: words it wul be painting, architecture, sculpture, words-in-freedom, music s. 
lines, and forms, a jumble of objects and reality thrown together at ran-. We 
shall offer new inspirations for the researches of painters, which will I to 
break out of the limits of the frame. We shall set in motion the words-in-i that 
smash the boundaries of literature as they march towards painting, :. noise- 
art, and throw a marvellous bridge between the word and the real 



.- will be: 

. C inematic analogies that use reality directly as one of the two elements of 
the analogy. Example: If we should want to express the anguished state of 
one of our protagonists, instead of describing it in its various phases of 
suffering, we would give an equivalent impression with the sight of ajagged 
and cavernous mountain. 

The mountains, seas, woods, cities, crowds, armies, squadrons, aero- 
planes, will often be our formidable expressive words: the universe will BE 
OUR vocabulary. Example: We want to give a sensation of strange 
cheerfulness: we show a chair cover flying comically around an enormous 
coat stand until they decide to join. We want to give the sensation of anger: 
we fracture the angry man into a whirlwind of little yellow balls. We want to 
give the anguish of a hero who has lost his faith and lapsed into a dead 
neutral scepticism: we show the hero in the act of matong an inspUTed 
speech to a great crowd; suddenly we bring on Giovanni Giolitti who trea- 
sonably stuffs a thick forkful of macaroni into the hero's mouth, drowning his 
winged words in tomalo sauce. 

We shall add colour to the dialogue by swiftly, simultaneously show-ing 
eveiy image that passes through the actors' brains. Example: repre- 
senting a man who will say to his woman: "You're as lovely as a gazelle," 
we shall show the gazelle. Example: if a character says, "I contemplate 
your fresh and luminous smile as a traveller after a long rough trip 
contemplates the sea from high on a mounlain," we shall show traveller, 
sea, mountain. 

This is how we shall make our characters as understandable as ifthey 
talked. 

2. Cinematic poems, speeches, and poetry. We shall make all of iheir com- 
ponenl images pass across ihe screen. 

Example: "Canto deN'amore" [Song of Love] by Giosue Carducci: 
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In their German strongholds perched Like 
falcons meditating the hunt 

We shall show the strongholds, the 
falcons in ambush. 

From the churches that raise long marble 

arms to heaven, in prayer to God 

From the convenis between villages and 

towns 

crouching darkly to the sound of bells 
like cuckoos among far-spaced trees 
singing boredoms and unexpected joys ... 

We shall show churches that little by little 
are changed into imploring women, God 
beaming down from on high, the convenis, 
the cuckoos, and so on. 

Example: "Sogno d'Estate" [Summer's 
Dream] by Giosue Carducci: 

Among your ever-sounding strains of battle, 
Homer, I am conquered by the warm hour: I 
bow my head in sleep on Scamander's 
bank, but my heart flees to the Tyrrhenian 
Sea. 

We shall show Carducci wandering amid 
the tumult of the Achaians, deftly avoiding 
the galloping horses, paying his respects 
to Homer, going for a drink with Ajax to 
the inn, The Red Scamander, and at the 
third glass of wine his heart, whose 
palpitations we ought to see, pops out of 
his jacket like a huge red balloon and flies 
over the Gulf of Rapallo. This is how we 
make 



films out of the most secret movements of 
genius. 



Thus we shall ridicule the works of the 
passeist poets, transforming to the great 
benefit of the public the most nostalgically 
monotonous weepy poetry into violent, 
exciting, and highly exhilarating 
spectacles. 



3. Cinematic simultanea y and intei pene 
tia t ion of different times and places. We 
shall project two or three diherent visual 
episodes at the sametime, one next to the 
other. 

4. Cinematic musical researches 
(dissonances, harmonies, symphonies of 
gestures, events, colours, lines, etc.). 

5. Dramatized states of mi nd on film. 

6. Daily exercises in freeing ourselves 
from mere photographed logic. 

7. Filmed dramas of objects. (Objects 
animated, humanized, baffled, dressedup, 
impassioned, civilized, dancing — objects 
removed from their normalsurroundings 
and put into an abnormal state that, by 
contrast, throws intorelief their amazing 
construction and nonhuman Ufe.) 
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8. Show windows of filmed ideas, events, 
types, objects, etc. 

9. Congr 
esses, flirts, fights and marriages of funny 
faces, mimicry, etc. 

Example: a big nose that silences a thousand 
congressional fingers by ring-ing an ear, while two 
policemen's moustaches arrest a tooth. 

10. Filmed 
unreal reconstructions of the human body. 

10. Filmed 
dramas of disproportion (a thirsty man who pulis 
out a tiny drink-ing straw that lengthens umbilically 
as far as a lake and dries it up instantly). 
10. Potenti 
al dramas and strategic plans of filmed feelings. 

10. Linear, 
plastic, chromatic equivalences, etc., of men, 
women, events, thoughts, music, feelings, weights, 
smells, noises (with white Unes on blackwe shall 
show the inner, physical rhythm of a husband who 
discovers hiswife in adultery and chases the lover — 
rhythm of soul and rhythm oflegs). 

11. Filmed 
words -in -freedom in movement (synoptic tables 
oflyric values — dramas of humanized or animated 
letters — orthographic dramas — typo-graphical 
dramas — geometric dramas — numeric sensibility, 
etc.). 

Painting + sculpture + plastic dynamism + 
words-in-freedom + corn-posed noises 
[intonarumori] + architecture + synthetic theatre = 
Futurist cinema. 

THIS IS HOW WE DECOMPOSE AND 
RECOMPUSE THE UNIVERSE ACCORDING TO 
OUR MARVELLOUS WHIMS, to centuple the powers of 
the Italian creative genius and its absolute pre- 
eminence in the world. 

11 SEPTEMBER1916 



— translated by R. W. flint 



"Thea ter, Circus, Variety" 
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« Founded jn Weimar, Germany, jn 1919, the Bauhaus School is best known 
*or its contribution to industrial design. But artists such as Laszlo Moholy- 
*Jagy, Oskar Schlemmer, Paul Klee, and Wassily Kandinsky undertook a wide 
range of aesthetic investigations, using the school as a laboratory to examine the 
-ormal principies of abstraction jn painting, photography, and sculpture. They z 
so explored the influence of technology, which had a profound jimpact on their 
Aork and ideas.These experiments led Moholy-Nagy and Schlemmer to develop a 
new kind of theater based on these principies. Their work was an attempt to 
synthesize the theater's essential components — space, composition, motion, 
sound, movement, and light — into a fully integrated, abstract form of artistic ex- 
pression. 

Moholy-Nagy referred to this jdea as the Theater of Totality, a reinterpreta-: 
on of Wagner's concept of "total theater." Moholy-Nagy's approach to the 
syn-: r esis of the arts reduced the importance of the written word and the 
presence : ; the actor, placing them on an equal plateau with stage design, 
lighting, music, and visual composition. His concept of the "Mechanized 
Eccentric" injected the qualities of machinery into every aspect of the stage 
performance, including the costume design, props, and movement of the actors, 
resulting in a theater that e-nphasized the physical rather than the literary, and 
that reflected the speed, dy-namism, and precision of state-of-the-art technology. 
It js notable that Moholy-Nagy integrated mechanical motifs in all his work — not 
only theater but also cainting, photography, film, and scuJpiure. 

In addition, Moholy-Nagy's theater challenged the relationship between the 
spectator and the performance. He called for the use of techniques that would 
alter the theatrical space, removing the traditional "fourth wall" that separated 
tne audience from the stage. He envisioned mechanical devices that would move 
across a multiplaned stage, a reorganizaron of the space that would literally im- 
nerse spectators jn the action. As Moholy-Nagy put it, the Bauhaus Theater 
would challenge the passivity of the audience, and "actually allow them to fuse 
with the action on the stage at the peak of cathartic ecstasy." » 



1 . the historical theater 

The historical theater was 
essentially a disseminator of 
information or propaganda, or 
it was an articulated 
concentrador! of action 
(Aktionskonzentration) de-rived 
from events and doctrines in 
their broadest meaning — that 
is to say, as 
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"dramatized" legend, as religious (cultist) or political (proselytizing) 
propaganda, or as compressed action with a more or less transparent 
purpose be-hind it. 

The theater dihered from the eyewitness report, simple storytelling, didactic 
moralizing, or advertising copy through its own particular synthesis of the 
elements of presentation: souND, color (light), motion, space, form (objects 

AND PERSONS). 

With these elements, in their accentuated but often uncontrolled interrela- 
tionships, the theater attempted to transmit an articulated experience. 

In early epic drama (Erzahlungsdrama) these elements were generally 
employed as illustration, subordinated to narration or propaganda. The next 
step in this evolution led to the drama of action (Aktionsdrama), where the 
elements of dynamic-dramatic movement began to crystallize: the theater of 
improvisaron, the commedia delirarte. These dramatic forms were 
progressively liberated from a central theme of logical, intellectual-emotional 
action which was no longer dominant. Gradually their moralizing and their 
tendentiousness disappeared in favor of an unhampered concentration on 
action: Shakespeare, the opera. 

With August Stramm, drama developed away from verbal context, from 
propaganda, and from character delineation, and toward explosive activism. 
Creative experiments with motion and sound (speech) were made, based on 
the Impetus of human sources of energy, that is, the "passions." Stramm's 
theater did not offer narrative material, but action and tempo, which, 
unpremeditated, sprang al-most automatically and in headlong succession 
from the human impulse for motion. But even in Stramm's case action was not 
altogether free from literary en-cumb ranee. 

"Literary encumbranee" is the result of the unjustifiable transfer of intellec- 
tualized material from the proper realm of literary effectiveness (novel, short 
story, etc.) to the stage, where it incorrectly remains a dramatic end in itself. 
The result is nothing more than literature if a reality or a potencial reality, no 
matter how imaginative is formulated or visually expressed without the 
creative forms peculiar only to the stage. It is not until the tensions 
concealed in the utmost economy of means are brought into universal and 
dynamic interaction that we have creative stagecraft (Buhnengestaltung). 



Even in recent times we have been deluded about the true value of creative 
stagecraft when revolutionary, social, eth-ical, or similar problems were 
unrolled with a great display of literary pomp and paraphernalia. 
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2. attempts at a theater form for 
today 

a) Theater ofSurprises: Futurists, 
Dadaists, Merz 

In the investigation of any morphology, 
we proceed today from the all-inclusive 
functionalism of goal, purpose, and 
materials. 

From this premisa the futurists, 
expressionists, and dadaists (merz) 
came to the conclusion that phonetic 
word relationships were more 
significan! than other creative literary 
means, and that the logical- 
intellectual conteni (das Logisch- 
Gedankliche) of a work of literature 
was far from its primary aim. It was 
maintained that, just as in 
representational painting it was not 
the conteni as such, not the objects 
represented which were essential, 
but the interaction of colors, so in 
literature it was not the logical- 
intellectual content which belonged in 
the foreground, but the effects which 
arose from the word-sound 
relationships. In the case of some 
writers this idea has been extended 
(or possibly con-tracted) to the point 
where word relationships are 



transformed into exclusively phonetic 
sound relationships, thereby totally 
fragmenting the word into con- 
ceptually disjointed vowels and 
consonants. 



theater which aimed at the 
elimination of logical-intellectual 
(literary) aspects. Yet in spite of this, 
man, who until then had been the 
sole representative of logical, causal 
action and of vital mental activities, 
still dominated. 



b) The Mechanized Eccentric (Die mechanische 
Exzentrik) 



As a logical consequence of this there arose the need for a MECHANIZED 
eccentric, a concentration of stage action in its purest form (erne 
Aktionskonzentration derBuhne in Reinkultur). Man, who no longer should be 
permitted to represen! himself as a phenomenon of spirit and mind through his 
intellectual and spiritual capacities, no longer has any place in this 
concentration of action. For, no mat-ter how cultured he may be, his organism 
permits him at best only a certain range of action, dependent entirely on his 
natural body mechanism. 

The effect of this body mechanism (Korpermechanik) (in circus performance 
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and athletic events, for example) arises essentially from the spectator's 
astonish-ment or shock at the potentialities of his own organism as 
demonstrated to him by others. This is a subjective effect. Here the human 
body is the sole medium of configuration (Gestaltung). For the purposes of 
an objective Gestaltung of movement this medium is limited, the more so 
since it has constant reference to sensible and perceptive (i.e., again literary) 
elements. The inadequacy of "human" Exzentrik led to the demand for a 
precise and fully controlled organi-zation of form and motion, intended to be a 
synthesis of dynamically contrasting phenomena (space, form, motion, sound, 
and light). This is the Mechanized Ec-centric. 



I 

3. the coming theater: theater of totality 



Every form process or Gestaltung has its general as well as its particular 
premises, from which it must proceed in making use of its specific media. We 
might, there-fore, clarify theater production (Theatergestaltung) if we 
investigated the nature of its highly controversial media: the human word and 
the human action, and. at the same time, considered the endless possibilities 
open to their creator — man. 

The origins of music as conscious composition can be traced back to the 
melodic recitations of the heroic saga. When music was systematized, 
permitting only the use of HARMONiES (klange) and excluding so-called sounds 
(gerausche). the only place left for a special sound form (Gerauschgestaltung) 
was in literatura, particularly in poetry. This was the underlying idea from which 
the Expression-ists, Futurists, and Dadaists proceeded in composing their 
sound-poem-(Lautgedichte). But today, when music has been broadened to 
admit sounds o: all kinds, the sensory-mechanistic effect of sound 
interrelationships is no longer a monopoly of poetry. It belongs, as much as do 
harmonies (Tone), to the realm of music, much in the same way that the task 
of painting, seen as color creation, is to organize clearly primary (apperceptive) 
color effect. Thus the error of the Futurists, the Expressionists, the Dadaists, 
and all those who built on such foun-dations becomes clear. As an example: 
the idea of an Exzentrik which is only mechanical. 

It must be said, however, that those ideas, in contradistinction to a literary- 
illustrative viewpoint, have unquestionably advanced creative theater 
precisely 
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because they were diametrically opposed. They canceled out the 
predominance of the exclusively logical-intellectual values. But once the 
predominance has been broken, the associative processes and the 
language of man, and consequently man himself in his totality as a formative 
medium for the stage, may not be barred from it. To be sure, he is no longer to 
be pivotal — as he is in traditional theater — but is to be employed on an 
equal footing with the other formative media. 

Man as the most active pnenomenon of life is indisputably one of the most 
ef-fective elements of a dynamic stage production (Buhnengestaltung), and 
there-fore he justifies on functional grounds the utilization of his totality of 
action, speech, and thought. With his intellect, his dialectic, his adaptability 
to any sit-uation by virtue of his control over his physical and mental powers, 
he is — when used in any concentration of action (Aktionskonzentration) — 
destined to be pri-marily a configuraron of these powers. 

And if the stage didn't provide him full play for these potentialities, it would 
be imperative to create an adequate vehicle. 

But this utilization of man must be clearly differentiated from his appearance 
heretofore in traditional theater. While there he was only the interpreter of a 
lit-erarily conceived individual or type, in the new theater of totality he will use 
the spiritual and physical means at his disposal PRODUCTIVELY and from his 
own INITIATIVE submit to the over-all action process. 

While during the Middle Ages (and even today) the center of gravity in 
theater production lay in the representation of the various types (hero, 
harlequin, peasant, etc.), it is the task of the FUTURE actor to discover and 
activate that which is common to all men. 



In the plan of such a theater the traditionally "meaningful" and causal 
intercon-nections can not play the major role. In the consideration of stage 
setting as an artform, we must learn from the creativa artist thatjust as it is 
impossible to ask what a man (as organism) is or stands for, it is inadmissible 
to ask the same ques-tion of a contemporary nonobjective picture which 
likewise is a Gestaltung, that is, an organism. 

The contemporary painting exhibits a multiplicity of color and surface in- 
terrelationships, which gain their effect, on the one hand, from their 
conscious and logical statement of problems, and on the other, from the 
unanalyzable intangibles of creative intuition. 
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In the same way, the Theater of Totality with its multifarious complexities c: 
light, space, plane, form, motion, sound, man — and with all the possibilities fi i 
varying and combining these elements — must be an ORGANISM. 

Thus the process of integrating man into creative stage production must be 
un-hampered by moralistic tendentiousness or by problems of science or the 
INDIVIDUAL. Man may be active only as the bearer of those functional elements 
which are organically in accordance with his specific nature. 
It is self-evident, however, that all other means of stage production must be 
given positions of effectiveness equal to man's, who as a living psychophysical 
or-ganism, as the producer of incomparable climaxes and infinite variations, 
de-mands of the conformative factors a high standard of quality. 

4. How shall the theater of totality be realized? 

One of two points of view still important today holds that theater is the concen- 
trated activation (Aktionskonzentration) of sound, light (color), space, form, 
and motion. Here man as coactor is not necessary, since in our day equipment 
can be constracted which is far more capable of executing the purdy 
mech&nical role of man than man himself. 

The other, more popular view will not relinquish the magnificent instrument 
which is man, even though no one has yet solved the problem of how to 
employ him as a creative medium on the stage. 

Is it possible to include his human, logical functions in a present-day con- 
centration of action on the stage, without running the risk of producing a 
copy from nature and without falling prey to Dadaist or Merz characterization, 
corn-posed of an eclectic patchwork whose seeming order is purely 
arbitrary? 

The creative arts have discovered pure media for their constructions: the 
primary relationships of color, mass, material, etc. But how can we integrate a 
se-quence of human movements and thoughts on an equal footing with the 
con-trolled, "absolute" elements of sound, light (color), form, and motion? In 
this regard only summary suggestions can be made to the creator of the new 
theater (Theatergestalter). For example, the repetition of a thought by many 
actors, with identical words and with identical or varying intonation and 
cadenee, could be employed as a means of creating synthetic (i.e., unifying) 
creative theater. 



"Theater, Circus, Variety," Theaterofthe Bauhaus 23 



(This would be the CHORUS — but not the attendant and passive chorus of 
antiq-uity!) Or mirrors and optica! equipment could be used to project the 
gigantically enlarged faces and gestures of the actors, while their voices could 
be amplified to correspond with the visual MAGNIFICATION. Similar effects can 
be obtained from the simultaneous, SYNOPTICAL, and synacoustical 
reproduction of thought (with motion pictures, phonographs, loud-speakers), 
or from the reproduction of thoughts suggested by a constructipn of variously 
meshing gears (eine zah-nradartig INEINANDERGREIFENDE 
Gedankengestaltung) '. 

Independen! of work in music and acoustics, the literature of the future will 
create its own "harmonies," at first primarily adapted to its own media, but with 
far-reaching implications for others. These will surely exercise an influence on 
the word and thought constructions of the stage. 

This means, among other things, that the phenomena of the subconscious 
and dreams of fantasy and reality, which up to now were central to the so 
called "intimate art theater" ("kammerspiele"), may no longer be predominant. 
And even if the conflicts arising from today's complicated social patterns, 
from the world-wide organization of technology, from paciiist-utopian and other 
kinds of revolutionary movements, can have a place in the art of the stage, they 
will be sig-nificant only in a transitional period, since their treatment belongs 
properly to the realms of literature, politics, and philosophy. 

We envision total stage action (gesamtbuhnenaktion) as a great dynamic- 
rhythmic process, which can compress the greatest clashing masses or 
accumu-lations of media — as qualitative and quantitative tensions — into 
elemental form. Part of this would be the use of simultaneously 
interpenetrating sets of contrast-ing relationships, which are of minor 
importance in themselves, such as: the tragicomic, the grotesque-serious, 
the trivial-monumental; hydraulic spectacles; acoustical and other "pranks"; 
and so on. Today's circus, operetta, vaude-ville, the clowns in America and 
elsewhere (Chaplin, Fratellini) have accom-plished great things, both in this 
respect and in eliminating the subjective — even if the process has been naive 
and often more superficial than incisive. Yet it would be just as superficial if we 
were to dismiss great performances and "shows" in this genre with the word 
Kitsch. It is high time to state once and for all that the much disdained 
masses, despite their "academic backwardness," often exhibit the 
soundest instincts and preferences. Our task will always remain the creative 
un-derstanding of the true, and not the imagined, needs. 
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5. the means 

Every Gestaltung or creative work should be an unexpected and new organ- 
ism, and it is natural and incumben! on us to draw the material for surprise 
ef-fects from our daily living. Nothing is more effective than the exciting new 
possibilities ofFered by the familiar and yet not properly evaluated elements ot 
modern life — that is, its idiosyncracies: individuation, classification, mecha- 
nization. With this in mind, it is possible to arrive at a proper understanding of 
stagecraft through an investigation of creative media other than man-as-actor 
himself. 

In the future, SOUND EFFECTS will make use of various acoustical equipmen: 
driven electrically or by some other mechanical means. Sound waves 
issuing from unexpected sources — for example, a speaking or singing are 
lamp, loud-speakers under the seats or beneath the floor of the auditorium, 
the use of new amplifying systems — will raise the audience's acoustic 
surprise-threshold so much that unequal effects in other areas will be 
disappointing. 

color (light) must undergo even greater transformation in this respect than 
sound. 

Developments in painting during the past decades have created the 
orgaru-zation of absolute color values and, as a consequence, the supremacy 
of pure and luminous chromatic tones. Naturally the monumentality and the 
lucid balance their harmonies will not tolerate the actor with indistinct or 
splotchy make-up and tattered costuming, a product of misunderstood 
Cubism, Futurism, etc. The use of precision-made metallic masks and 
costumes and those of various other composition materials will thus become 
a matter of course. The pallid face, t:.: subjectivity of expression, and the 
gestures of the actor in a colored stage envi-ronment are therefore 
eliminated without impairing the effective contrast be-tween the human 
body and any mechanical construction. Films can also be projected onto 
various surfaces and further experiments in space illuminati will be devised. 
This will constitute the new action of light, which by mean» of modern 
technology will use the most intensified contrasts to guarantee it A r a position 
of importance equal to that of all other theater media. We have not y; begun to 
realize the potential of light for sudden or blinding illumination, for &^J-effects, 
for phosphorescent effects, for bathing the auditorium in light synchro-ni/ed 
with climaxes or with the total extinguishing of lights on the stage. All this. 



"Theater, Circus, Variety," Theater o'i the Bauhaus 25 



of course, is thought of in a sense totally difieren! from anything in current tra- 
dirional theater. 

From the time that stage objects became mechanically movable, the generally 
traditional, horizontally structured organization of movement in space has 
been enriched by the possibility of vertical motion. Nothing stands in the way of 
mak-ing use of complex APPARATUS such as film, automobile, elevator, 
airplane, and odier machinery, as well as optical instruments, reflecting 
equipment, and so on. The current demand for dynamic construction will be 
satisfied in this way, even though it is still only in its first stages. 

There would be a further enrichment if the present isolation of the stage 
could be eliminated. In today's theater, stage and spectator are too much 
separated, too obviously divided into active and passive, to be able to produce 
creative re-lationships and reciprocal tensions. 

It is time to produce a kind of stage activity which will no longer permit the 
masses to be silent spectators, which will not only excite them inwardly but will 
let them take hold and participate — actually allow them to fuse with the action 
on the stage at the peak of cathartic ecstasy. 

To see that such a process is not chaotic, but that it develops with control 
and organization, will be one of the tasks of the thousand-eyed new director, 
equipped with all the modern means of understanding and communication. 

It is clear that the present peep-show stage is not suitable for such 
organized motion. 

The next form of the advancing theater — in cooperation with future authors — 
will probably answer the above demands with suspended bridges and draw- 
bridges running horizontally, diagonally, and vertically within the space of the 
theater; with platform stages built far into the auditorium; and so on. Apart from 
rotating sections, the stage will have movable space constructions and dislike 
areas, in order to bring certain action moments on the stage into prominence, 
as in film "close-ups." In place of today's periphery of orchestra loges, a 
runway joined to the stage could be built to establish — by means of a more or 
less caliper-like embrace — a closer connection with the audience. 

The possibilities for a variation of levels of movable planes on the stage of 
the future would contribute to a genuine organization of space. Space will 
then no longer consist of the interconnections of planes in the oid meaning, 
which was able to conceive of architectonic delineation of space only as an 
en- 
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closure formed by opaque surfaces, The new space originales from free-standing 
surfaces or from linear definition of planes (wire frames, antennas), so that the 
surfaces stand at times in a very free relationship to one another, without the 
need of any direct contad. 

As soon as an intense and penetrating concentration of action can be function- 
ally realized, there will develop simultaneously the corresponding auditorium AR- 
chitecture. There will also appear costumes designed to emphasize function and 
costumes which are conceived only for single moments of action and capa-ble of 
sudden transformations. 

There will arise an enhanced control over all formative media, unified in a har- 
monious effect and built into an organism of perfect equilibrium. 
— translated by arthur S. wensinger 

Dick Higgins 



"Intermedia" (1966) 
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Dick Higgins (secondfrom left) performing Philip Corner's "Piano 
Activities." Courtesy ofArchiv Sohm, Staatsgalerie Stuttgart. 



"The Happen'mg developed as intermedium, an uncharted land that lies 
between collage, muslo, and the theater. It js not governed by rules; each 
work determines jts own medium and form according to its needs. " 
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« Fluxus artist Dick Hlggins describes a branch of American postmodernism 
from the 1960s that reflects the tumultuous social atmosphere of the era and jts 
impact on the arts. Higgins's attention is focused on intermedia, a myriad of 
emerging genres that spilled across the boundaries of traditional media. In the 
interstices between the arts, mixed-media forms coalesced: Happenings, 
performance art, kinetic sculpture, electronic theater, as well as a variety of 
deliberately uncategorizable works. 

Higgins cites Marcel Duchamp's readymades, including the jnfamous urinal 
Fountain, asthefirst challenge to the European tradition of the sacred aura of the 
artwork. Following the example of Duchamp, Higgins suggests that artists 
explore the territory that lies between "the general area of art media and those of 
life media." He callsfor unusual combinations of art, including, for instance, the 
mixing of painting and shoes (as jn the art of Claes Oldenberg). With 
intermedia, any available object or experience can be incorporated into the 
artwork. This was the notion at the core of the Happening, a form of performance 
pio-neered by Oldenberg, Alian Kaprow, Jim Dine, and Robert Whitman that 
was the pinnacle of intermedia art. Their works took place in staged 
environments, presented as collages of action, music, and found objects that the 
audience encountered jn an order typically without linear development, and 
without the through line of traditional narrative. 

The art of "mixed-means" was defined by jts tendency to establish an open 
system without rules or boundaries.This open-endedness allowed Higgins and his 
colleagues to extend the Happening to incorporate the audience as an additional, 
composed element of the work; as he put it: "uve people as part of the collage." 
This shift jn the role of the spectator had profound implications for the evolution 
of interactive media art, and its influence is felt in the changing role of the viewer 
in human-computer interaction. » 

IVLIIch. of the best work being produced today seems to fall between 
media. This is no accident. The concept of the separation between media 
arose in the Renaissance. The idea that a painting is made of paint on canvas 
or that a sculpture should not be painted seems characteristic of the kind of 
social thought — cat-egorizing and dividing society into nobility with its various 
subdivision)), untitled gentry, artisans, serfs and landless workers — which we 
cali the feudal conception of the Great Chain of Being. This essentially 
mechanistic approach continued to 
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be relevant throughout the first two industrial revolutions, just concluded, and 
into the present era of automation, which constitutes, in fact, a third industrial 
revolution. 

However, the social problems that characterize our time, as opposed to 
the political ones, no longer allow a compartmentalized approach. We are 
ap-proaching the dawn of a classless society, to which separation into rigid 
categories is absolutely irrelevant. This shift does not relate more to East than 
West or vice versa. Castro works in the cane fields. New York's Mayor Lindsay 
walks to work during the subway strike. The millionaires eat their lunches at 
Horn and Hardart's. This sort of populism is a growing tendency rather than a 
shrinking one. 

We sense this in viewing art which seems to belong unnecessarily rigidly to 
one or another form. We view paintings. What are they, after all? Expensive, 
handmade objects, intended to ornament the walls of the rich or, through 
their (or their government's) munificence, to be shared with the large numbers 
of peo-ple and give them a sense of grandeur. But they do not allow any 
sense of dialogue. 

Pop art? How could it play a part in the art of the future? It is bland. It is 
pure. It uses elements of common life without comment, and so, by accepting 
the mis-ery of this life and its aridity so mutely, it condones them. Pop and Op 
are both dead, however, because they confine themselves, through the media 
which they employ, to the older functions of art, of decorating and 
suggesting grandeur, whatever their detailed content or their artists' 
intentions. None of the ingenious theories of the Mr. Ivan Geldoway combine 
can prevent them from being colos-sally boring and irrelevant. Milord runs his 
Mad Avenue gallery, in which he dis-plays his pretty wares. He is protected by 
a handful of rude footmen who seem to feel that this is the way Life will 
always be. At his beck and cali is Sir Fretful Callous, a moderately well- 
informed high priest, who apparently despises the Fiame he is supposed to 
tend and therefore prefers anything that thillates him. However, Milord needs 
his services, since he, poor thing, hasn't the time or the energy to contribute 
more than his name and perhaps his dollars; getting infor-mation and finding 
out what's going on are simply toooooo exhausting. So, well protected and 
advised, he goes blissfully through the streets in proper Louis XlVstyle. 

This scene is just not characteristic of the painting world as an institution, 
however. It is absolutely natural to (and inevitable in) the concept of the pure 
medium, the painting or precious object of any kind. That is the way such 
objects are marked since that is the world to which they belong and to which 
they relate. 
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The sense of "I am the state," however, will shortly be replaced by "After ine 
the deluge," and, in fact, if the High Art world were better informed, it would 
real-ize that the deluge has already begun. 

Who knows when it began? There is no reason for us to go into history in 
any detail. Part of the reason that Duchamp's objects are fascinating while 
Picasso's voice is fading is that the Duchamp pieces are truly between 
media, between sculpture and something else, while a Picasso is readily 
classifiable as a painted ornament. Similarly, by invading the land between 
collage and photography, the German John Heartfield produced what are 
probably the greatest graphics of our century, surely the most powerful political 
art that has been done to date. 

The ready-made or found object, in a sense an intermedium since it was 
not intended to conform to the pure media, usually suggests this, and 
therefore sug-gests a location in the field between the general area of art media 
and those of life media. However, at this time, the locations of this sort are 
relatively unexplored, as compared with media between the arts. I cannot, for 
example, name work that has consciously been placed in the intermedium 
between painting and shoes. The closest thing would seem to be the 
sculpture of Claes Oldenburg, which falls between sculpture and hamburgers 
or Eskimo Pies, yet it is not the sources of these images themselves. An 
Oldenburg Eskimo Pie may look something like an Eskimo Pie, yet it is neither 
edible or cold. There is still a great deal to be done in this direction in the way 
of opening up aesthetically rewarding possibilities. 

In the middle 1950s many painters began to realize the fundamental irrele- 
vance of Abstract Expressioaistu, which was Ore domirant mode at the 
time. Such painters as Alian Kaprow and Robert Rauschenberg in the United 
States and Wolf Vostell in Germany turned to collage or, in the latter's case, 
de-collage in the sense of making work by adding or removing, replacing and 
substituting or altering components of a visual work. They began to include 
increasingly in-congruous objects in their work. Rauschenberg called his 
constructions combines and went so far as to place a stuffed goat — spattered 
with paint and a rubber tire around its neck — onto one. Kaprow, more 
philosophical and restless, medi-tated on the relationship of the spectator and 
the work. He put mirrors into his things so the spectator could feel included in 
them. That wasn't physical enough, so he made enveloping collages which 
surrounded the spectator. These he called environments. Finally, in the spring 
of 1958, he began to include live people as part of the collage, and this he 
called a happening. 

The proscenium theater is the outgrowth of seventeenth-century ideais of 
social order. Yet there is remarkably little structural difference between the 
dramas of D'Avenant and those of Edward Albee, certainly nothing comparable 
tothedif- 
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ference in pump construction or means of mass transportation. It would 
seem that the technological and social implications of the first two industrial 
revolu-tions have been evaded completely. The drama is still mechanistically 
divided: there are performers, production people, a separate audience and 
an explicit script. Once started, like Frankenstein's monster, the course of 
affairs is unalter-able, perhaps damned by its inability to reflect its 
surroundings. With our pop-ulistic mentality today, it is difficult to attach 
importance — other than what we have been taught to attach — to this 
traditional theater. Nor do minor innovations do more than provide dinner 
conversation: this theater is round instead of square, in that one the stage 
revolves, here the play is relatively senseless and whimsical (Pinter is, after 
all, our modern J. M. Barrie — unless the honor be-longs more properly to 
Beckett). Every year fewer attend the professional Broad-way theaters. The 
shows get sillier and sillier, showing the producers' estimate of our mentality 
(or is it their own that is revealed?). Even the best of the traditional theater 
is no longer found on Broadway but at the Judson Memorial Church, some 
miles away. Yet our theater schools grind out thousands on thou-sands of 
performing and production personnel, for whom Jobs will simply not exist in 
twenty years. Can we blame the unions? Or rents and real estate laxes? Of 
course not. The subsidized productions, sponsored at such museums as New 
York's Lincoln Center, are not building up a new audience so much as 
reculti-vating an oid one, since the medium of such drama seems weird and 
artificial in our new social milieu. We need more portability and flexibility, and 
this the traditional theater cannot provide. It was made for Versailles and for 
the sedentary Milords, not for motorized life-demons who travel six hundred 
miles a week. Versailles no longer speaks very loudly to us, since we think at 
eighty-five miles an hour. 

In the other direction, starting from the idea of theater itself, others such 
as myself declared war on the script as a set of sequential events. 
Improvisation was no help; performers merely acted in imitation of a script. 
So I began to work as if time and sequence could be utterly suspended, not 
by ignoring them (which would simply be illogical) but by systematically 
replacing them as structural el-ements with change. Lack of change would 
cause my pieces to stop. In 1958 I wrote a piece, Stacked Deck, in which any 
event can take place at any time, as long as its cue appears. The cues are 
produced by colored lights. Since the colored lights could be used wherever 
they were put, and audience reactions were also cuing situations, the 
performance-audience separation was removed and a hap-pening situation 
was established, though less visually oriented in its use of its en-vironment 
and imagery. At the same time, Al Hansen moved into the area from 
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graphic notation experiments, and Nam June Paik and Benjamin Patterson 
(both in Germany at the time) moved in from varieties of music in which 
specifically musical events were frequently replaced by nonmusical actions. 

Thus the Happening developed as an intermedium, an uncharted land 
that lies between collage, music, and the theater. It is not governed by rules; 
each work determines its own medium arid form according to its needs. The 
concept itself is better understood by what it is not, rather than what it is. 
Approaching it, we are pioneers again, and shall continue to be so as long 
as there's plenty of elbow room and no neighbors around for a few miles. Of 
course, a concept like this is very disturbing to those whose mentality is 
compartmentalized. Time, Life and the High Priests have been announcing 
the death of Happenings, reg-ularly since the movement gained momentum in 
the late fifties, but this says more about the accuracy of their information than 
about the liveliness of the movement. 

We have noted the intermedia in the theater and in the visual arts, the 
Happening, and certain varieties of physical constructions. For reasons of 
space we cannot take up here the intermedia between other areas. However, 
I would like to suggest that the use of intermedia is more or less universal 
throughout the fine arts, since continuity rather than categorization is the 
hallmark of our new mentality. There are parallels to the Happening in music, 
for example, in the work of such composers as Philip Corner and John Cage, 
who explore the intermedia between music and philosophy, or Joe Jones, 
whose self-playing musical instru-ments fall into the intermedium between 
music and sculpture. The constructed music of Emmet Williams and Robert 
Filliou certainly constitute an intermedium between poetry and sculpture. Is it 
possible to speak of the use of intermedia as a huge and inclusive movement 
of which dada, futurism, and surrealism are early phases preceding the huge 
ground-swell that is taking place now? Or is it more reasonable to regard the 
use of intermedia an irreversible historical innovation, more comparable to 
the development of instrumental music than, for example, to the 
development of romanticism? 
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Jean Tinguely, Homage to New York. Photo by David Gahr. 
Courtesy of Billy Kliiver. 



"The new interface I will define is one \n which the artist makes active use 
ofthe inventiveness and skills of an engineer to achieve his purpose. The 
artist could not complete his intentions without the help of an engineer. The 
artist incorporales the work ofthe engineer in the painting or the sculpture 
or the performance. " 
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« In the late 1950s, the Swedish-born engineer Billy Kluver worked on laser 
systems for Bell Laboratories jn Murray Hill New Jersey. Kluver, however, was not 
satisfied by purely scientific pursuits, and his interests led him to explore the 
artistic milieu that gave rise to pop art, minimalism, and Happenings a short drive 
away in New York City. Kluver befriended the leading exponents of the newarts, 
which led tocollaborationswith such key figures as Robert Rauschenberg, Jasper 
Johns, Andy Warhol, and John Cage. He became the chief catalyst for the art and 
technology movement that was launched dramatically jn the spring of 1960, at 
the Museum of Modern Art, with Jean Tinguely's infamous self-destructing kj- 
netic sculpture, Homageto New York. Kluver's participaron jn this work, with its 
paint bombs, chemical stinks, noisemakers, and fragments of scrap metal, inspired 
a generation of artists to imagine the possibilities of technology, as the machine 
destroyed itself, in Kluver's words, "jn one glorious act of mechanical suicide." 

In the 1960s, Kluver, more than anyone, sawthe potential for the integration 
of art and technology. Inspired by Aristotle's notion of Techne — in which there 
was no differentiation between the practice of art and science — Kluver proposed 
the active and equal participation of the artist and engineer in the creation of the 
artwork. In this collaboration, he believed that the engineer required the partic- 
ipation of the artist, who, as a "visionary about life" and an active agent of 
social change, involved the engineer jn meaningful cultural dialogue. At the 
same time, he felt that the artist, jn the spirit of Rauschenberg's famous credo "to 
ciose the gap between art and life," had an obligation to incorporate technology 
as an element jn the artwork, since technology had become inseparable from our 
uves. 

Shortly after writing this article, Kluver cofounded the now legendary Ex- 
periments in Art and Technology (E.A.T.) with Rauschenberg. E.A.T.encouraged 
the collaboration of artists and engineers across the country jn interdisciplinary 
technology-based art projects. Kluver's work takes on increasing significance as 
the notion of the synthesis of art and technology has assimilated into the con- 
temporary arts. » 

Well, to begin with, the title of my talk is not going to be entirely 
unrelated to what I am going to say. What I will discuss is a new mode of 
interaction between science and technology on the one hand and art and life 
on the other. To use a scientific jargon that is currently in, I will try to define a 
new interface between these two areas. 
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Technology has always been closely tied in to the development of art. For 
Aristotle, Techne means both art and technology. As they became different 
sub-jects they still fed on each other. New technological discoveries were taken 
up and used by artists and you are all familiar with the contributions of artists 
to technology. The contemporary artist reads with ease the technical trade 
magazines. The new chemical material is hardly developed before it gets 
used by an artist. Today the artist tends to adopt the new material or the new 
industrial process as his insignia. We talk about artists in terms that he works 
in such and such a way or that he uses such and such materials. We hear 
about artists being poisoned and hurt in their work. In this century, artists 
have also embraced technology as sub-ject matter: the enthusiasm of the 
Futurists, the experiments of Dada, the opti-mism of the Bauhaus 
movement and the Constructivists, all have looked at technology and 
science and found material for the artists. But for all this inter-est, art 
remains a passive viewer of technology. Art has only been interested in the 
fallout, so to speak, of science and technology. The effect of technology on art 
can apparently be even a negative one: the invention of the camera helped kill 
off rep-resentational painting, and we are now witnessing how the computer 
is about to take care of music and non-representational painting. 

The new interface I will define is one in which the artist makes active use 

of the inventiveness and skills of an engineer to achieve his purpose. The 

artist 

could not complete his intentions without the help of an engineer. The artist 
in- A uipuidics uic worK of tne engineer in the painting or the sculpture or 
the performance. A characteristic of this kind of interaction is that 
generally only one work of art results. In other words, the engineer is 
notjust inventing a new and special process for the use of the artist. He 
does notjust teach the artist a new skill which the artist can use to extract 
new aesthetic variations. Technology is well aware of its own beauty and 
does not need the artist to elaborate on this. I will argue that the use of 
the engineer by the artist is not only unavoidable but nec-essary. 

Before I try to justify why I believe that this interface exists and why the 
interaction between artists and engineers will become stronger, let me 
give you a few simple examples of what I mean in terms of works that 
already exist. I shall be modest and limit myself to use examples from my 
own experience. But there exist several others. 

You probably have heard about Jean Tinguely's self-destroying 
machine, Hommage to New York, which more or less destroyed itself on 
March 17,1960, in the sculpture garden of the Museum of Modern Art in 
New York. In retrospect I thirik my modest contribution to the machine 
was to visit garbage dumps in 



36 BillyKluver 



New Jersey to pick up bicycle wheels and to truck them to 53rd Street. 
However, there were a few technical ideas hidden in Tinguely's machine which 
incidentally were mainly the contributions of my technical assistant at the 
time, Harold Hodges. There were about eight electrical circuits in the 
machine which closed successively as the machine progressed toward its 
ultimate fate. Motors would start, smoke would come out, smaller machines 
would leave the big one to escape. In order to make the main structure 
collapse, Harold had devised a scheme using supporting sections of Wood's 
metal which would melt from the heat of over-heated resistors. At another 
point this method was used to light a candle. Con-trary to what I hear 
frequently said, Tinguely's machine did not contain mam o: these technical 
links. It was mostly Tinguely's motors that did it. 

A better example is two neon light power supplies that we made for two 
paintings by Jasper Johns. In one case, the light was the letter A, in the other 
the letter R. What was new was that Johns wanted no cords to the painting. To 
stack up batteries to 1200 volts would have been messy, dangerous and 
impractical. Se we started out with 12 volts of rechargeable batteries and 
devised a multivibra-tor circuit which, together with a transformer, would 
give us 1200 volts. The technical equipment, all 400 dollars' worth of it, was 
mounted behind John's painting. 

My final example is Rauschenberg's large sculpture Oracle, which was 
shown in New York last year. It was the result of work carried out over three 
years dur-ing which time two complete technical systems were fimshed and 
junked. The final system enables the sound from five AM radios to be heard 
from each of the five sculptures in the group, but with each radio being 
controlled from a central control unit, in one of the sculptures. There are no 
connecting wires between the sculptures and they are all freely movable, on 
wheels. 

All these examples have one thmg in common: they are ndiculous from an 
en-gineer's point of view. Why would anyone want to spend 9000 dollars to be 
abic to control five AM radios simultaneously, in one room? I want to 
emphasize that the examples contain very simple engineering and should 
not be taken as ven' original. But each of the projects required an engineer or 
a technically skilled per-son to achieve what the artist wanted. And an 
important point is that the artiv could not be quite sure about the outcome. 
We have been taught by Robert Rauschenberg that the painting is an objeci 
among other objects, subjected to the same psychological and physical influ- 
ences as other objects. During a musical piece byjohn Cage, we are forced to 
ac-cept the equality of all the sounds we hear as part of the composition. In 
his happenings, Claes Oldenburg lets the actors play themselves although in 
most in- 
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stances the actors are unaware of this. He writes his Happenings with a 
particular person in mind, allowing the specific shyness, nervousness, 
sensuality of the person to become part of the Happening. The tradition in art 
can, therefore, not tell us anything else but that the technical elements 
involved in the works I have described are just as much a part of the work of 
art as the paint in the painting. It is impossible to treat the sound as part oi 
Oracle and not the radios. Jasper Johns has already shown us the backside of 
the canvas and I am afraid he will have to accept the not-so-elegant backsides 
oiField Painting and Zone as well. But if the radios and the amplifiers are part 
of the work — what about the engineer who designs them? In the same way as 
Oldenburg works with the peculiarities of people in his Happenings, the artist 
has to work with the peculiarities and the for-eign mode of operation of the 
engineer. On the basis of this observation, I hereby declare myself to be a work 
of art — or rather an integral part of the works of art I have just described. I am 
definitely not a violin player who interprets and feels for the work of his master. I 
know nothing about art or the artists involved. I am an engineer and as such, 
only raw material for the artist. 

But how can I claim that this new interface between art and technology 
does in fact exist? Maybe I wanted to become a work of art and devised this 
inge-nious scheme for my own ends? Well, I think that we don't have to look 
too far. We all know how technology has become part of our lives. And now 
we can see absolutely no reason why it should not become more so. No 
sound has been heard from another culture to oppose Western technology. 
The faster the un-derdeveloped countries can have it, the faster they want it. 
On the other end of the spectrum, we now have systems where we don't 
know quite where the machine ends and the human being begins. I am 
thinking of the space program which has introduced the new and maybe 
inhuman objective: the system has to work, no failures are allowed, no 
personal emotions or mistake may interfere with the success of the project. 
The space program is developing a new man-agerial type which is totally 
responsible. I read recently that President Johnson has let the contract to 
solve the Appalachian problem go to the electronics in-dustry. We are now 
getting the fallout from Cape Kennedy and can expect more. 
The great initiator of all this technological soul-searching is the computer. La- 
boriously we are translating every aspect of human activity into computer lan- 
guage. In fact, I believe the computer will turn out to be the greatest 
psychoanalyst of all times. Now where does all this leave us? The engineers 
may be psychoanalysts but they are not visionaries. John Cage has recently 
written a wonderful article called "How to Improve the World." As a blind 
engineer, one of his observations gave me a realjolt. Cage points out that there 
exist systems of 
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interaction between human beings which work without any police or power 
structure whatsoever. In fact, there are hundreds of agreements between 
the countries of the world that work perfectly well. In particular, technological 
ques-tions are dealt with without any complications. It seems that technology 
breeds agreement. This is such a simple observation that it frightens you that 
you did not think of it. I believe that Cage's discovery fullyjustifies the 
statement that technology will force the solution of such problems as food 
distribution and housing. There is no other stable optimum but to give 
people food and housing. The Dadaists' suggestion of free food and 
Buckminster Fuller's suggestion of free housing for the people of the world 
will happen. But the alternatives that the en-gineer can imagine for the full use 
of the fantastic capacity of technology are even so few and limited. He is, as I 
said, no visionary about life. But the artist is a visionary about life. Only he can 
create disorder and still get away with it. Only he can use technology to its 
fullest capacity. John Cage has suggested: Let the engi-neer take care of 
order and art (in the traditional sense) and let the artists take care of disorder 
and life. And I am adding technology. This to sum up: First the artists have to 
create with technology because technology is becoming inseparable from our 
lives. "Technology is the extension of our nervous system," as McLuhan 
says. Second, the artists should use technology because technology needs 
the artists. Technology needs to be revealed and looked at — much like we 
undress a 
woman. 

The artist's work is like that of a scientist. It is an investigation which may or 
may not yield meaningful results; in many cases we only know many years 
later. What I am suggesting is that the use of the engineer by the artist will 
stimulate new ways of looking at technology and dealing with life in the 
future. 

What about power failure? I wish we knew more about what happened. 
We heard a lot about how people became friendly and helped each other 
out. The whole thing could have been an artist's idea — to make us aware of 
something. In the future there will exist technological systems as complicated 
and as large as the Northeastern power grid whose sole purpose will be to 
intensify our lives through increased awareness. 
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\amJunePaik, Opera Sextronique. Photo by Ludwig Winterhalter. 
Courtesy ofNam June Paik. 

"There js no rewind button on the Betamax oflife. " 
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« The Korean born Nam June Paik's formative years were grounded jn the 
study of rigorous post-Webern, twelve-tone music composition. After meeting the 
iconoclastic American composer John Cage at the Darmstadt Summer School 
of Music jn 1958, Paik turned to performance art and mixed-media works that 
challenged his classical roots. Most important, Paik embraced the medium of 
television, and became the founding father of video art. H is long and prolific re- 
lationship with electronic media began notably with the cellist Charlotte Moor- 
man, jn controversial performance works such as Opera Sextronique (1967). 
Paik's oeuvre later included television sculpture, satellite art, robotic devices, 
and giant video walls with synthesized imagery pulsating from stacks of cathode- 
ray tubes. 

In the brief, mischievously elfin manifesto "Cybernated Art," Paik suggests 
that art should embrace the technologies of an information society. Paik presents 
himself as artist-shaman, synthesizing art and technology in an effort to exor- 
cise the demons of a mass-consumer, technology-obsessed society. Paik uses re- 
jected media artifacts in his work, such as vintage television sets; his video works, 
with their liberal doses of "cybernated shock and catharsis," are poignantly cyn- 
ical pieces that comment on an American tech no-culture dominated by starry- 
eyed optimists. 

In the latter essay, "Art and Satellite," he explains that the material of the 
information age — the raw data, the continuousflow of conteni — is becoming an 
intrinsic element of his artwork. He sees in the satellite, the most advanced com- 
municatjqns technoloqy, of its day A an opportunity, to connect minds and encour- 
age new ways of thinking. Through the connectivity of a "cybernated society," a 
work of art can become dynamic, always changing, as data flows through the 
wired network and across our screens'. As the artwork incorporates the unend- 
ing flow and restless nature of information itself, transcending geographical 
boundaries, jt brings about a "synthesis" of all cultures across borders. » 

cybernated art (1966) 

Cybernated art is very importan!, but art for cybernated life is more impor- 
tant, and the latter need not be cybernated. (Maybe George Brecht's 
simplissimo is the most adequate.) 

» But if Pasteur and Robespierre are right that we can resist poison only 
through certain built-in poison, then some specific frustrations, caused by 
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cybernated life, require accordingly cybernated shock and catharsis. My 
everyday work with videotape and the cathode-ray tube convinces me of 
this. 

Cybernetics, the science of pure relations, or relationship itself, has its ori- 
gin in karma. Marshall McLuhan's famous phrase "Media is message" 
was formulated by Norbert Wiener in 1948 as "The signal, where the 
message is sent, plays equally important role as the signal, where 
message is not sent." 

As the Happening is the fusion of various arts, so cybernetics is the ex- 
ploitation of boundary regions between and across various existing sci- 
ences. 

" Newton's physics is the mechanics of power and the unconciliatory two- 
party system, in which the strong win over the weak. But in the 1920s a 
German genius put a tiny third-party (grid) between these two mighty 
poles (cathode and anode) in a vacuum tube, thus enabling the weak to 
win over the strong for the first time in human history. It might be a 
Buddhistic "third way," but anyway this German invention led to 
cybernetics, which carne to the world in the last war to shoot down 
German planes from the English sky. 

The Buddhists also say 

Karma is samsara 

Relationship is metempsychosis 

We are in open circuits 



art and satellite (1984) 

At the turn of our century, the French mathematician Henri Poincare said the 
fol-lowing thing.... (Yes, it was in the midst of so-called material progress and 
the 

discovery of new Things ) Poincare pointed out that what was being discov- 
ered was not new things but merely the new relationships between things al- 
ready existing. 

We are again in the fin de siecle . . . this time we are discovering much nCW 
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software ... which are not new things but new thinks ... and again we are 
dis-covering and even weaving new relationships between many thinks and 
minds... we are already knee-deep in the post industrial age. The satellite, 
especially the 

live two-way satellite, is a very powerful tool for this human Videosphere 

It is said that all the sciences can trace their roots to Aristotle: but the 
science of cosmic aesthetics started with Sarutobi Sasuke, a famous ninja 
(a samurai who mastered many fantastic arts, including that of making 
himself invisible, chiefly to spy upon an enemy). The first step for a ninja. 
is learning how to shorten distances by shrinking the earth, that is, how to 
transcend the law of gravity. For the satellite, this is a piece of cake. Sojust 
as Mozart mastered the newly invented clarinet, the satellite artist must 
compose his art from the begin-ning suitable to physical conditions and 
grammar. Satellite art in the superior sense does not merely transmit 
existing symphonies and operas to other lands. It must consider how to 
achieve a two-way connection between opposite sides of the earth; how to 
give a conversational structure to the art; how to master dif-ferences in time; 
how to play with improvisation, in-determinism, echos, feed-backs, and 
empty spaces in the Cagean sense; and how to instantaneously manage the 
differences in culture, preconceptions, and common sense that exist between 
various nations. Satellite art must make the most of these elements (for they 
can become strengths or weaknesses), creating a multitemporal, 
multispatial sym-phony.... 

There is no rewind button on the BETAMAX of life. An important event takes 
place only once. The free deaths (of Socrates, Christ, Bo Yi and Shu Qi) that 
be-came the foundations for the morality of three civilizations occurred only 
once. The meetings of person and person, of person and specific era are 
often said to take place "one meeting-one life," but the bundle olsegments of 
this existence (if segments can come in bundles) has grown much thicker 
because of the satellite. The thinking process is the jumping of electrical 
sparks across the synapses between brain cells arranged in multilayered 
matrices. Inspiration is a spark shooting offin an unexpected direction and 
landing on a point in some corner of the matrix. The satellite will 
accidentally and inevitably produce unexpected meetings of person and 
person and will enrich the synapses between the brain cells of mankind. 
Thoreau, the author of Walden, Life in the Woods, and a nineteenth- 
century forerunner of the hippies, wrote, "The telephone company is trying 
to connect Maine and Tennessee by telephone. Even if it were to suc-ceed, 
though, what would the people say to each other? What could they pos- 
sibly find to talk about?" Of course, history eventually answered 
Thoreau's questions (silly ones, at that). There developed a feedback (or, to 
use an older 
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term, dialectic) of new contacts breeding new contents and new contents 
breeding new contacts.... 

Thanks to the satellite, the mysteries of encounters with others (chance 
meet-ings) will accumulate in geometric progression and should become 
the main nonmaterial product of post-industrial society. God created love to 
propagate the human race, but, unawares, man began to love simply to love. 
By the same logic, although man talks to accomplish something, unawares, 
he soon begins to talk simply to talk.... 



part II 



Interactivity 



Norbert Wiener 



"Cybernetics in History," The Human Use of Human 
Beings (1954) 



Norbert Wiener atMIT. Courtesy ofMIT Computer 
Museum. 

"Soc/efy can only be understood through a study ofthe messages and the 
communication faduties which belong to it; and that \n the Mure development of these 
messages and communication facilities, messages between man and machines, 
between machines and man, and between machine and machine, are destined to play 
an ever increasing part. " 




47 



48 Norbert Wiener 



« Noi-bert Wiener, a Hungarian-born mathematician, electrical engineer, and 
Communications specialist, wrote about the nature of human-machine commu- 
nication jn his landmark treatise of 1948, Cybernetics. In an attempt to make his 
theories on information science and machine control accessible to a broader 
public, two years later he published The Human Use of Human Beings, drawing f rom 
sociology to theorize about how people might interact and coexist with comput- 
ers, then revised the book for a new edition in 1954. 

Wiener claimed that the quality of man-machine communication influences 
man's innerwell-being. Histheory of cybernetics was meantto improve the quality 
of life jn a technological society, where people are increasingly reliant on 
machines, and where interactions with machines are the norm. 

Wiener defined cybernetics as the science of transmitting messages between 
man and machine, or from machine to machine.The term cybernetics has its roots 
in the Greek word for "steersman" or "governor," and Wiener's use of it suggests 
how people interact with machines through a controlling device, such as a steer- 
ing mechanism. Driving a car, passing through an automatic door, or clicking a 
mouse are all cybernetic actions. Wiener's remarkable insight, which is the 
premise behind all human-computer interactivity and interface design, js that 
human communication should be a model for human-machine and machine-to- 
machine interactions. Wiener points out that the quality of datatransmission 
between man and machine is affected by such factors as feedback, noise, and 
entropy. As he succinctly puts it, "the fact that the signal in its intermediate stages 
has gone through a machine rather than through a person is irrelevant." » 

OMICe the end of World War II, I have been working on the many ramifications 
of the theory of messages. Besides the electrical engineering theory of the trans- 
mission of messages, there is a larger field which includes not only the study of 
language but the study of messages as a means of controlling machinery and 
society, the development of computing machines and other such automata, 
certain reflections upon psychology and the nervous system, and a tentative new 
theory of scientific method. This larger theory of messages is a probabilistic 
theory, an intrinsic part of the movement that owes its origin to Willard Gibbs. 

Until recently, there was no existing word for this complex of ideas, and in 
order to embrace the whole field by a single term, I felt constrained to invent one. 
Henee "Cybernetics," which I derivad from the Greek word kubernetes, or 
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"steersman," the same Greek word from which we eventually derive our word 
"governor." Incidentally I found later that the word had already been used by 
Ainpere with reference to political science, and had been introduced in 
another context by a Polish scientist, both uses dating from the earlier part of 
the nine-teenth century. 

I wrote a more or less technical book entitled Cybernetics which was pub- 
lished in 1948. In response to a certain demand for me to make its ideas 
accept-able to the lay public, I published the first edition of The Human Use of 
Human Beings in 1950. Since then the subject has grown from a few ideas 
shared by Drs. Claude Shannon, Warren Weaver, and myself into an 
established region of re-search. Therefore, I take this opportunity occasioned 
by the reprinting of my book to bring it up to date, and to remove certain 
defects and inconsequentiali-ties in its original structure. 

In giving the definition of Cybernetics in the original book, I classed com- 
munication and control together. Why did I do this? When I communicate with 
another person, I impart a message to him, and when he communicates 
back with me he returns a related message which contains information 
primarily ac-cessible to him and not to me. When I control the actions of 
another person, I communicate a message to him, and although this 
message is in the imperative mood, the technique of communication does not 
differ from that of a message of fact. Furthermore, if my control is to be 
effective I must take cognizance of any messages from him which may 
indicate that the order is understood and has been obeyed. 

It is the thesis of this book that society can only be understood through a 
study of the messages and the communication facilities which belong to it; 
and that in the future development of these messages and communication 
facilities, messages between man and machines, between machines and 
man, and between machine and machine are destined to play an ever- 
increasing part. 

When I give an order to a machine, the situation is not essentially different 
from that which arises when I give an order to a person. In other words, as far 
as my consciousness goes I am aware of the order that has gone out and of 
the sig-nal of compliance that has come back. To me, personally, the fact that 
the signal in its intermediate stages has gone through a machine rather than 
through a person is irrelevant and does not in any case greatly change my 
relation to the signal. Thus the theory of control in engineering, whether 
human or animal or mechanical, is a chapter in the theory of messages. 

Naturally there are detailed differences in messages and in problems of 
control, not only between a living organism and a machine, but within each 
nar- 
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rower class of beings. It is the purpose of Cybernetics to develop a language 
and techniques that will enable us indeed to attack the problem of control and 
com-munication in general, but also to find the proper repertory of ideas and 
techniques to classify their particular manifestations under certain concepts. 

The commands through which we exercise our control over our environ- 
ment are a kind of information which we impart to it. Like any form of informa- 
tion, these commands are subject to disorganization in transit. They 
generally come through in less coherent fashion and certainly not more 
coherently than they were sent. In control and communication we are always 
fighting nature's tendency to degrade the organized and to destroy the 
meaningful; the tendency, as Gibbs has shown us, for entropy to increase. 

Much of this book concerns the limits of communication within and among 
individuals. Man is immersed in a world which he perceives through his sense 
or-gans. Information that he receives is coordinated through his brain and 
nervous system until, after the proper process of storage, collation, and 
selection, it emerges through effector organs, generally his muscles. These in 
turn act on the external world, and also react on the central nervous system 
through receptor organs such as the end organs of kinaesthesia; and the 
information received by the kinaesthetic organs is combined with his already 
accumulated store of information to influence future action. 

Information is a name for the content of what is exchanged with the outer 
world as we adjust to it, and make our adjustment felt upon it. The process of 
re-ceiving and of using information is the process of our adjusting to the 
contin-gencies of the outer environment, and of our living effectively within 
that environment. The needs and the complexity of modern life make 
greater de-mands on this process of information than ever before, and our 
press, our mu-seums, our scientific laboratories, our universities, our libraries 
and textbooks, are obliged to meet the needs of this process or fail in their 
purpose. To live effectively is to live with adequate information. Thus, 
communication and control belong to the essence of man's inner life, even as 
they belong to his life in society.... 

Messages are themselves a form of pattern and organization. Indeed, it 
is possible to treat sets of messages as having an entropy like sets of states of 
the external world. Just as entropy is a measure of disorganization, the 
information carried by a set of messages is a measure of organization. In fact, 
it is possible to interpret the information carried by a message as essentially the 
negative of its entropy, and the negative logarithm of its probability. That is, the 
more probable the message, the less information it gives. Cliches, for example, 
are less illuminating 

than great poems. . . . 



"Cybernetics jn History," The Human Use of Human Beings 51 

Let us consider the activity of the little figures which dance on the top of a 
music box. They move in accordance with a pattern, but it is a pattern which 
is set in advance, and in which the past activity of the figures has practically 
noth-ing to do with the pattern of their future activity. The probability that they 
will diverge from this pattern is nil. There is a message, indeed; but it goes 
from the machinery of the music box to the figures, and stops there. The 
figures them-selves have no trace of communication with the outer world, 
except this one-way stage of communication with the preestablished 
mechanism of the music box. They are blind, deaf, and dumb, and cannot 
vary their activity in the least from the conventionalized pattern. 

Contrast with them the behavior of man, or indeed of any moderately intel- 
ligent animal such as a kitten. I cali to the kitten and it looks up. I have sent it 
a message which it has received by its sensory organs, and which it registers 
in ac-tion. The kitten is hungry and lets out a pitiful wail. This time it is the 
sender of a message. The kitten bats at a swinging spool. The spool swings to 
its left, and the kitten catches it with its left paw. This time messages of a very 
complicated nature are both sent and received within the kitten's own nervous 
system through certain nerve end-bodies in its joints, muscles, and tendons; 
and by means of nervous messages sent by these organs, the animal is aware 
of the actual position and tensions of its tissues. It is only through these organs 
that anything like a manual skill is possible. 

I have contrasted the prearranged behavior of the little figures on the 
music box on the one hand, and the contingent behavior of human beings 
and animais on the other. But we must not suppose that the music box is typical 
of all machine behavior. 

The older machines, and in particular the older attempts to produce 
automata, did in fact function on a closed clockwork basis. But modern 
automatic machines such as the controlled missile, the proximity fuse, the 
automatic door opener, the control apparatus for a chemical factory, and the 
rest of the modern armory of automatic machines which perform military or 
industrial functions possess sense organs; that is, receptors for messages 
coming from the outside. These may be as simple as photoelectric cells 
which change electrically when a light falls on them, and which can tell light 
from dark, or as complicated as a television set. They may measure a tension 
by the change it produces in the con-ductivity of a wire exposed to it, or they 
may measure temperature by means of a thermocouple, which is an 
instrument consisting of two distinct metais in con-tact with one another 
through which a current flows when one of the points of contact is heated. 
Every instrument in the repertory of the scientific-instrument 
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maker is a possible sense organ, and may be made to record its reading 
remotely through the intervention of appropriate electrical apparatus. Thus 
the machine which is conditioned by its relation to the external world, and by 
the things hap-pening in the external world, is with us and has been with us 
for some time. 

The machine which acts on the external world by means of messages is 
also familiar, The automatic photoelectric door opener is known to every 
person who has passed through the Pennsylvania Station in New York, and is 
used in many other buildings as well. When a message consisting of the 
interception of a beam of light is sent to the apparatus, this message actuales 
the door, and opens it so that the passenger may go through. 

The steps between the actuation of a machine of this type by sense organs 
and its performance of a task may be as simple as in the case of the electric 
door; or it may be in fact of any desired degree of complexity within the limits of 
our en-gineering techniques. A complex action is one in which the data 
introduced, which we cali the input, to obtain an effect on the outer world, 
which we cali the output, may involve a large number of combinations. These 
are combinations, both of the data put in at the moment and of the records 
taken from the past stored data which we cali the memory. These are 
recorded in the machine. The most complicated machines yet made which 
transform input data into output data are the high-speed electrical computing 
machines, of which I shall speak later in more detail. The determination of 
the mode of conduct of these machines is given through a special sort of 
input, which frequently consists of punched cards or tapes or of magnetized 
wires, and which determines the way in which the machine is going to act in 
one operation, as distinct from the way ir. which it might have acted in another. 
Because of the frequent use of punched or magnetic tape in the control, the 
data which are fed in, and which indicate the mode of operation of one of 
these machines for combining information, are calleo the taping. 

I have said that man and the animal have a kinaesthetic sense, by which 
thev keep a record of the position and tensions of their muscles. For any 
machine subject to a varied external environment to act effectively it is 
necessary that information concerning the results of its own action be furnished 
to it as part of the information on which it must continue to act. For example, if 
we are running ar. elevator, it is not enough to open the outside door because 
the orders we have given should make the elevator be at that door at the time 
we open it. It is in:-portant that the release for opening the door be 
dependent on the fact that ir.-. elevator is actually at the door; otherwise 
something might have detained it. ar.: the passenger might step into the empty 
shaft. This control of a machine on is.- 
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basis of its actual performance rather dian its expected performance is known 
as feedback, and involves sensory members which are actuated by motor 
members and perform the function of tell-tales or monitors — that is, of elements 
which indicate a performance. It is the function of these mechanisms to 
control the me-chanical tendency toward disorganization; in other words, to 
produce a temporary and local reversal of the normal direction ofentropy. 

I have just mentioned the elevator as an example of feedback. There are 
other cases where the importance of feedback is even more apparent. For 
example, a gun-pointer takes information from his instruments of observation, 
and conveys it to the gun, so that the latter will point in such a direction that 
the missile will pass through the moving target at a certain time. Now, the 
gun itself must be used under all conditions of weather. In some of these the 
grease is warm, and the gun swings easily and rapidly. Under other 
conditions the grease is frozen or mixed with sand, and the gun is slow to 
answer the orders given to it. If these orders are reinforced by an extra push 
given when the gun fails to respond easily to the orders and lags behind 
them, then the error of the gun-pointer will be de-creased. To obtain a 
performance as uniform as possible, it is customary to put into the gun a 
control feedback element which reads the lag of the gun behind the position it 
should have according to the orders given it, and which uses this dif-ference 
to give the gun an extra push. 

It is true that precautions must be taken so that the push is not too hard, 
for if it is, the gun will swing past its proper position, and will have to be pulled 
back in a series of oscillations, which may well become wider and wider, and 
lead to a disastrous instability. If the feedback system is itself controlled — if, 
in other words, its own entropic tendencies are checked by still other 
controlling mechanisms — and kept within limits sufficiently stringent, this will 
not occur, and the existence of the feedback will increase the stability of 
performance of the gun. In other words, the performance will become less 
dependent on the frictional load; or what is the same thing, on the drag 
created by the stiffness of the grease. 

Something very similar to this occurs in human action. If I pick up my cigar, 
I do not will to move any specific muscles. Indeed in many cases, I do not 
know what those muscles are. What I do is to turn into action a certain 
feedback mech-anism; namely, a reflex in which the amount by which I have 
yet failed to pick up the cigar is turned into a new and increased order to the 
lagging muscles, whichever they may be. In this way, a fairly uniform voluntary 
command will en-able the same task to be performed from widely varying 
initial positions, and ir-respective of the decrease of contraccion diie to fatigue 
of the muscles. Similarly, when I drive a car, I do not follow out a series of 
commands dependent simply 
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on a mental image of the road and the task I am doing. If I fmd the car 
swervingtoo much to the right, that causes me to pull it to the left. This 
depends on theactual performance of the car, and not simply on the road; 
and it allows me todrive with nearly equal efficiency a light Austin or a heavy 
truck, without havingformed separate habits for the driving of the two 

It is my thesis diat the physical functioning of the living individual and the op- 
eration of some of the newer communication machines are precisely parallel 
in their analogous attempts to control entropy through feedback. Both of them 
have sensory receptors as one stage in their cycle of operation: that is, in both 
of them there exists a special apparatus for collecting information from the 
outer world at low energy leveis, and for making it available in the operation of 
the individual or of the machine. In both cases these external messages are 
not taken neat, but through the internal transforming powers of the 
apparatus, whether it be alive or dead. The information is then turned into a 
new form available for the fur-ther stages of performance. In both the animal 
and the machine this performance is made to be effective on the outer world. 
In both of them, their performed ac-tion on the outer world, and not merely 
their intended action, is reported back to the central regulatory apparatus. This 
complex of behavior is ignored by the av-erage man, and in particular does not 
play the role that it should in our habitua-analysis of society; for just as 
individual physical responses may be seen from this point of view, so may the 
organic responses of society itself. I do not mean that the sociologist is 
unaware of the existence and complex nature of Communications in society, 
but until recently he has tended to overlook the extent to whicr. they are the 
cement which binds its fabric together. 

We have seen in this chapter the fundamental unity of a complex of ideas 
which until recently had not been sufficiently associated with one another, 
namely, the contingent view of physics that Gibbs introduced as a modifica 
tio: of the traditional, Newtonian conventions, the Augustinian attitude toward 
order and conduct which is demanded by this view, and the theory of the 
messair among men, machines, and in society as a sequence of events in 
time which- j though it itself has a certain contingency, strives to hold back 
nature's tendency toward disorder by adjusting its parts to various purposive 
ends. 
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J.C.R. Licklider. Courtesy ojMIT Computer Museum.. 

"The hope js that, in not too many years, human bra'ms and computing machines will be 
coupled together very tightiy and that the resulting partnership will think as no human 
brain has ever thought and process data in a way not approached by the information- 
handling machines we know today. " 
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« Inthe late 1950s, J.C.R, Licklider was one of the few scientists who sawthe 
computer's potential as a collaborative partner in the creative process. In his 
book Libraries of the Future he proposed how computers networked jn a system 
could lead to a new kind of "thinking center." During his tenure at the United 
States government's Advanced Research Projects Agency (ARRA) in the 1960s, 
Licklider had the vision to support controversial but critical research that led to 
the rise of human-computer interactivity and the personal computer. Lickiider's 
own research on computer networking was a catalyst for the creation of the 
Internet (originally known as "ARPANET") in 1969. 

With this article, Licklider fundamentally changed howwe interact with 
computers by proposing the novel jdea of a symbiotic relationship between man 
and machine.The computer's then typical role was as a subservient device that 
per-formed data operations and mechanical calculations. Licklider suggested 
that a computer could be more effective as a collaborator, and that this 
interaction would yield results well beyond what people could achieve on their 
own. He saw the potential for a dialogue between man and machine, a symbiotic 
partnership that would unleash tremendous creative potential, made possible by 
the ease, im-mediacy, and flexibility of a keyboard and real-time graphics display. 
This fore-sight was extraordinary considering that "Man-Computer Symbiosis" 
was written when computers were excruciatingly slow and clumsy, with 
mainframe systems using punch card input and teletype output the norm. » 



SUMMARY 

Man-computer symbiosis is an expected development in cooperative 
interaction between men and electronic computers. It wul involve very ciose 
coupling between the human and the electronic members of the partnership. 
The main aims are (1) to let computers facilitate formulative thinking as they 
now facilitate the so-lution of formulated problems, and (2) to enable men and 
computers to cooper-ate in making decisions and controlling complex 
situations without inflexible dependence on predetermined programs. In the 
anticipated symbiotic partnership, men will set the goals, formulate the 
hypotheses, determine the criteria, and perform the evaluations. Computing 
machines will do the routinizable work that must be done to prepare the way 
for insights and decisions in technical and sci-entific thinking. Preliminary 
analyses indicate that the symbiotic partnership will perform intellectual 
operations much more effectively than man alone can perform them. 
Prerequisites for the achievement of the effective, cooperative association 
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include developments in computer time sharing, in memory componente, in mem- 
ory organization, in programming languages, and in input and output equipment. 



1 introduction 

1.1 Symbiosis 

The fig tree is pollinated only by the insect Blastophaga grossorun. The larva of 
the insect lives in the ovary of the fig tree, and there it gets its food. The tree and 
the insect are thus heavily interdependent: the tree cannot reproduce without the 
insect; the insect cannot eat without the tree; together, they constitute not only a 
viable but a productive and thriving partnership. This cooperative "living 
together in intimate association, or even ciose union, of two dissimilar organisms" 
is called symbiosis. 1 

"Man-computer symbiosis" is a subclass of man-machine systems. There 
are many man-machine systems. At present, however, there are no man-computer 
symbioses. The purposes of this paper are to present the concept and, hopefully, 
to foster the development of man-computer symbiosis by analyzing some prob- 
lems of interaction between men and computing machines, calling attenu'on to 
applicable principles of man-machine engineering, and pointing out a few ques- 
tions to which research answers are needed. The hope is that, in not too many 
years, human brains and computing machines will be coupled together very 
tightly, and that the resulting partnership will think as no human brain has ever 
thought and process data in a way not approached by the information-handling 
machines we know today 

1.2 Between "Mechanically Extended Man" and "Artificial Intelligence" 

As a concept, man-computer symbiosis is different in an important way from 
what North 2 has called "mechanically extended man." In the man-machine 
systems of the past, the human operator supplied the initiative, the direction, the 
in-tegration, and the criterion. The mechanical parts of the systems were mere 
extensions, first of the human arm, then of the human eye. These systems cer- 
tainly did not consist of "dissimilar organisms living together..." There was only 
one kind of organism — man — and the rest was there only to help him. 
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In one sense of course, any man-made system is intended to help man, to 
help a man or men outside the system. If we focus upon the human operator 
within the system, however, we see that, in some areas of technology, a 
fantastic change has taken place during the last few years. "Mechanical 
extension" has given way to replacement of men, to automation, and the men 
who remain are there more to help than to be helped. In some instances, 
particularly in large computer-centered information and control systems, the 
human operators are responsible mainly for functions that it proved infeasible 
to automate. Such systems ("hu-manly extended machines," North might cali 
them) are not symbiotic systems. They are "semi-automatic" systems, 
systems that started out to be fully auto-matic but fell short of the goal. 

Man-computer symbiosis is probably not the ultimate paradigm for com- 
plex technological systems. It seems entirely possible that, in due course, 
elec-tronic or chemical "machines" will outdo the human brain in most of 
the functions we now consider exclusively within its province. Even now, 
Gelernter's IBM-704 program for proving theorems in plane geometry proceeds 
at about the same pace as Brooklyn high school students, and makes similar 
errors. 3 There are, in fact, severa! theorem-proving, problem-solving, chess- 
playing, and pattern-recognizing programs (too many for complete reference) 4 
capable of rivaling human intellectual performance in restricted areas; and 
Newell, Simon, and Shaw's 5 "general problem solver" may remove some of 
the restrictions. In short, it seems worthwhile to avoid argument with (other) 
enthusiasts for artificial in-telligence by conceding dominance in the distant 
future of cerebration to machines alone. There will nevertheless be a fairly 
long interim during which the main intellectual advances will be made by men 
and computers working together in intimate association. A multidisciplinary 
study group, examining future re-search and development problems of the 
Air Force, estimated that it would be 1980 before developments in artificial 
intelligence make it possible for machines alone to do much thinking or 
problem solving of military significance. That would leave, say, five years to 
develop man-computer symbiosis and 15 years to use it. The 15 may be 10 or 
500, but those years should be intellectually the most creative and exciting in 
the history of mankind. 



2 aims of man-computer symbiosis 

Present-day computers are designed primarily to solve preformulated 
problems or to process data according to predetermined procedures. The 
course of the 
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computation tnay be conditional upon results obtained during the 
computation, but all the alternatives must be foreseen in advance. (If an 
unforeseen alternative arises, the whole process comes to a halt and awaits 
the necessary extension of the program.) The requirement for preformulation 
or predetermination is some-times no great disadvantage. It is often said that 
programming for a computing machine forces one to think clearly, that it 
disciplines the thought process. If the user can think his problem through in 
advance, symbiotic association with a computing machine is not necessary. 

However, many problems that can be thought through in advance are very 
dif-ficult to think through in advance. They would be easier to solve, and they 
could be solved faster, through an intuitively guided trial-and-error procedure in 
which the computer cooperated, turning up flaws in the reasoning or 
revealing unex-pected turns in the solution. Other problems simply cannot be 
formulated with-out computing-machine aid. Poincare anticipated the 
frustration of an important group of would-be computer users when he said, 
"The question is not, 'What is the answer?' The question is, 'What is the 
question?' " One of the main aims of man-computer symbiosis is to bring the 
computing machine effectively into the formulative parts of technical 
problems. 

The other main aim is closely related. It is to bring computing machines 
effectively into processes of thinking that must go on in "real time," time that 
moves too fast to permit using computers in conventional ways. Imagine 
trying, for ex-ample, to direct a battle with the aid of a computer on such a 
schedule as this. You formulate your problem today. Tomorrow you spend with 
a programmer. Next week the computer devotes 5 minutes to assembling 
your program and 47 sec-onds to calculating the answer to your problem. You 
get a sheet of paper 20 feet long, full of numbers diat, instead of providing a 
final solution, only suggest a tac-tic that should be explored by simulation. 
Obviously, the battle would be over be-fore the second step in its planning 
was begun. To think in interaction with a computer in the same way that you 
think with a colleague whose competence supplements your own will require 
much tighter coupling between man and machine than is suggested by the 
example and than is possible today. 



3 need for computer participation in formulative and real- 
time thinking 



The preceding paragraphs tacitly made the 
assumption that, if they could be in-troduced 
effectively into the thought process, the 
functions that can be per- 
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formed by data-processing machines would improve or facilitate thinking and 
problem solving in an importan! way. That assumption may require justifica- 
tion. 

3.1 A Preliminary and Informal Time-and-Motion Analysis of Technical 
Thinking 

Despite the fact that there is a voluminous literature on thinking and problem 
solving, including intensive case-history studies of the process of invention, I 
could find nothing comparable to a time-and-motion-study analysis of the 
mental work of a person engaged in a scientific or technical enterprise. In the 
spring and summer of 1957, therefore, I tried to keep track of what one 
moderately technical person actually did during the hours he regarded as 
devoted to work. Although I was aware of the inadequacy of the sampling, I 
served as my own subject. 

It soon became apparent that the main thing I did was to keep records, 
and the project would have become an infinite regress if the keeping of 
records had been carried through in the detail envisaged in the initial plan. It 
was not. Nev-ertheless, I obtained a picture of my activities that gave me 
pause. Perhaps my spectrum is not typical — I hope it is not, but I fear it is. 

About 85 percent of my "thinking" time was spent getting into a position to 
think, to make a decision, to learn something I needed to know. Much more 
time went into finding or obtaining information than into digesting it. Hours went 
into the plotting of graphs, and other hours into instructing an assistant how to 
plot. When the graphs were finished, the relations were obvious at once, but 
the plotting had to be done in order to make them so. At one point, it was 
necessary to compare six experimental determinations of a function 
relating speech-intelligibility to speech-to-noise ratio. No two experimenters 
had used the same definition or measure of speech-to-noise ratio. Several 
hours of calculating were required to get the data into comparable form. 
When they were in comparable form, it took only a few seconds to determine 
what I needed to know. 

Throughout the period I examined, in short, my "thinking" time was devoted 
mainly to activities that were essentially clerical or mechanical: searching, 
calculating, plotting, transforming, determining the logical or dynamic conse- 
quences of a set of assumptions or hypotheses, preparing the way for a 
decision or an insight. Moreover, my cholees of what to attempt and what not 
to attempt were determined to an embarrassingly great extent by 
considerations of clerical feasibility, not intellectual capability. 
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The main suggestion conveyed by the findings just described is that the 
operations that fill most of the time allegedly devoted to technical thinking are 
operations that can be performed more effectively by machines than by men. 
Severe problems are posed by the fact that these operations have to be 
performed upon diverse variables and in unforeseen and continually changing 
sequences. If those problems can be solved in such a way as to create a 
symbiotic relation between a man and a fast information-retrieval and data- 
processing machine, however, it seems evident that the cooperative interaction 
would greatly improve the thinking process. 

It may be appropriate to acknowledge, at this point, that we are using the temi 
"computer" to cover a wide class of calculating, data-processing, and 
information-storage-and-retrieval machines. The capabilities of machines in this 
class are increasing almost daily. It is therefore hazardous to make general state- 
ments about capabilities of the class. Perhaps it is equally hazardous to make 
general statements about the capabilities of men. NevertheJess, certain 
genotypic differences in capability between men and computers do stand out, 
and they have a bearing on the nature of possible man-computer symbiosis 
and the po-tential value of achieving it. 

As has been said in various ways, men are noisy, narrow-band devices, but 
their nervous systems have very many parallel and simultaneously active chan- 
nels. Relative to men, computing machines are very fast and very accurate, but 
they are constrained to perform only one or a few elementary operations at a 
time. Men are flexible, capable of "programming themselves contingently" on the 
basis of newly received information. Computing machines are single-minded, 
constrained by their "pre-programming." Men naturally speak redundan! lan- 
guages organized around unitary objects and coherent actions and employing 20 
to 60 elementary symbols. Computers "naturally" speak nonredundant lan- 
guages, usually with only two elementary symbols and no inherent appreciation 
either of unitary objects or of coherent actions. 

To be rigorously correct, those characterizations would have to include many 
qualifiers. Nevertheless, the picture of dissimilarity (and therefore potential sup- 
plementation) that they present is essentially valid. Computing machines can do 
readily, well, and rapidly many things that are difficult or impossible for man, and 
men can do readily and well, though not rapidly, many things that are difucult or 
impossible for computers. That suggests that a symbiotic cooperation, if suc- 
cessful in integrating the positive characteristics of men and computers, would be 
of great value. The differences in speed and in language, of course, pose diffi- 
culties that must be overeome. 
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4 separable functions of men and computers in the anticipated 

symbiotic association 

It seems likely that the contributions of human operators and equipment will 
blend together so completely in many operations that it will be difficult to 
separate them neatly in analysis. That would be the case if, in gathering data on 
which to base a decision, for example, both the man and the computer carne up 
with rel-evant precedents from experience and if the computer then suggested 
a course of action that agreed with the man's intuitive judgment. (In theorem- 
proving programs, computers fmd precedents in experience, and in the SAGE 
System, they suggest courses of action. The foregoing is not a far-fetched 
example.) In other operations, however, the contributions of men and 
equipment will be to some extent separable. 

Men will set the goals and supply the motivations, of course, at least in the 
early years. They will formulate hypotheses. They will ask questions. They will 
think of mechanisms, procedures, and models. They will remember that such- 
and-such a person did some possibly relevant work on a topic of interest back in 
1947, or at any rate shortly after World War II, and they will have an idea in what 
journals it might have been published. In general, they will make approximate 
and fallible, but leading, contributions, and they will define criteria and serve as 
evaluators, judging the contributions of the equipment and guiding the general 
lineofthought. 

In addition, men will handle the very-low-probability situations when such 
situations do actually arise. (In current man-machine systems, that is one of the 
human operator's most important functions. The sum of the probabilities of 
very-low-probability alternatives is often much too large to neglect.) Men will ful in 
the gaps, either in the problem solution or in the computer program, when the 
computer has no mode or routine that is applicable in a particular circumstance. 

The information-processing equipment, for its part, will convert hypotheses 
into testable models and then test the models against data (which the human 
operator may designate roughly and identify as relevant when the computer pre- 
sents them for his approval). The equipment will answer questions. It will 
simulate the mechanisms and models, carry out the procedures, and display 
the results to the operator. It will transform data, plot graphs ("cutting the caite" 
in whatever way the human operator specifies, or in several alternative ways if 
the 
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human operator is not sure what he wants). The equipment will interpolate, 
extrapolate, and transform. It will convert static equations or logical 
statements into dynamic models so the human operator can examine their 
behavior. In general, it will carry out the routinizable, clerical operations that fill 
the intervals be-tween decisions. 

In addition, the computer will serve as a statistical-inference, decision-theory, 
or game-theoiy machine to make elementary evaluations of suggested 
courses ofaction whenever there is enough basis to support a formal statistical 
analysis. Fi-nally, it will do as much diagnosis, pattern-matching, and 
relevance-recognizingas it profitably can, but it will accept a clearly secondary 
status in those areas 
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Douglas Engelbart 

"Augmenting Human Intellect: A Conceptual 
Framework" (1962) 



Douglas Engelbart at the Augmentation Research Center. Courtesy of 
Douglas Engelbart, Bootstrap Institute. 

"The term 'intelligence amplificacion' seems applicable to our goal 
ofaugmenting the human intellect jn that the entity to be produced will exhibit 
more of what can be called inteIHgence than an unaided human could; we will 
have amplified the inteIHgence ofthe human by organizing his intelectual 
capabilities into higher leveis of synergistic structuring." 
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« Douglas Engelbart js one of the most influential thinkers jn the history of 
personal computing. He is best known as the groundbreaking engineer who in- 
vented such mainstays of the personal computer as the mouse, Windows, email, and 
the word processor. He has received less recognition for his pioneering con-cepts 
behind the interactive computer networks that are the basis for the Internet. 
Engelbart led one of the most important projects funded by ARPA in the 1960s: a 
networked environment designed to support collaborative interaction between 
people using computers. It was dubbed the oNLine System (NLS).This historie 
prototype, developed atthe Augmentaron Research Center of the Stan-ford 
Research Institute (SRI) and unveiled jn 1968 at the Fall Joint Computer 
Conference jn San Francisco, influenced the development of the first personal 
computer and the graphical user interface at Xerox PARC in the early 1970s. 

Engelbart's experience as a radar specialist in wartime led him to consider the 
potential for a computer connected to a monitor display and an input device (such 
as a keyboard), which, jn the fifties and sixties, transformed the way people 
interacted with computers. In this paper, Engelbart describes in detail how such a 
computer might function — a description that predicts in remarkable detail the 
fundamental qualities of a modern personal computer. The linking of people and 
computers using this approach to interactivity would result jn the use of computers 
to "solve the world's problems," Engelbart wrote, by boosting the capacities of the 
mind's intellectual faculties. 

Engelbart reasoned that networked computing would not only make individuals 
more intellectually effective; it would enable a collaborative method of sharing 
knowledge. At the time, the jdea that computers could be operated intu-itively by 
teams engaged jn creative work was highly unorthodox. Engelbart pro-posed a 
language, process, methodology, and conceptual framework for the real-time 
interaction of collaborative computing. He was among the first to consider using 
computer networks to create collective knowledge bases among groups of 
professionals.This approach has since transformed many of our insti-tutions, 
including the military, universities, museums, and libraries, as they have extended 
their reach into cyberspace. Engelbart's theories about the power of collective 
knowledge via a real-time electronic medium have also led to other forms of 
many-to-many human-computer interaction, including such collective online 
activities as virtual communities, on-line forums, and teleconferencing. » 
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I. introduction 
A. General 

By "augmenting human intellect" we mean increasing the capability of a man 
to approach a complex problem situation, to gain comprehension to suit his 
particular needs, and to derive solutions to problems. Increased capability in 
this re-spect is taken to mean a mixture of the following: more-rapid 
comprehension, better comprehension, the possibility of gaining a useful 
degree of comprehension in a situation that previously was too complex, 
speedier solutions, better solutions, and the possibility of finding solutions to 
problems that before seemed insoluble. And by "complex situations" we 
include the professional problems of diplomais, executives, social scientists, 
life scientists, physical scientists, attor-neys, designers — whether the 
problem situation exists for twenty minutes or twenty years. We do not speak 
of isolated clever tricks that help in particular situations. We refer to a way of life 
in an integrated domain where hunches, cut-and-try, intangibles, and the 
human "feel for a situation" usefully co-exist with powerful concepts, 
streamlined terminology and notation, sophisticated meth-ods, and high- 
powered electronic aids. 

Man's population and gross product are increasing at a considerable rate, 
but the complexity of his problems grows still faster, and the urgency with which 
solutions must be found becomes steadily greater in response to the 
increased rate of activity and the increasingly global nature of that activity. 
Augmenting man's intellect, in the sense defined above, would warrant full 
pursuit by an enlightened society if there could be shown a reasonable 
approach and some plausible ben-efits. 

This report covers the first phase of a program aimed at developing means 
to augment the human intellect. These "means" can include many things — all 
of which appear to be but extensions of means developed and used in the 
past to help man apply his native sensory, mental, and motor capabilities — 
and we con-sider the whole system of a human and his augmentation means 
as a proper field of search for practical possibilities. It is a very important 
system to our society, and like most systems its performance can best be 
improved by considering Die whole as a set of interacting components rather 
than by considering the compo-nents in isolation. 

This kind of system approach to human intellectual effectiveness does not 
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find a ready-made conceptual framework such as exists for established 
disciplines. Befo re a research program can be designed to pursue such an 
approach intelligently, so that practical benefits might be derived within a 
reasonable time while also producing results of long-range significance, a 
conceptual framework must be searched out — a framework that provides 
orientation as to the important factors of the system, the relationships among 
these factors, the types of change among the system factors that offer likely 
improvements in performance, and the sort of research goals and methodology 
that seem promising. 1 

In the first (search) phase of our program we have developed a conceptual 
framework that seems satisfactory for the current needs of designing a 
research phase. Section II contains the essence of this framework as derived 
from several different ways of looking at the system made up of a human and 
his intellect-augmentation means. 

The process of developing this conceptual framework brought out a number 
of significan! realizations: that the intellectual effectiveness exercised today by a 
given human has little likelihood of being intelligence limited — that there are 
dozens of disciplines in engineering, mathematics, and the social, life, and phys- 
ical sciences that can contribute improvements to the system of intellect- 
augmentation means; that any one such improvement can be expected to 
trigger a chain of coordinating improvements; that until every one of these 
disciplines comes to a standstill and we have exhausted all the improvement 
possibilities we could glean from it, we can expect to continue to develop 
improvements in this "human-intellect" system; that there is no particular 
reason not to expect gains in personal intellectual effectiveness from a 
concerted system-oriented approach that compare to those made in personal 
geographic mobility since horseback and sailboat days.... 

To give the reader an initial orientation about what sort of thing this 
computer-aided working system might be, we include below a short description 
of a possible system of this sort. This illustrative example is not to be considered 
a description of the actual system that will emerge from the program. It is given 
only to show the general direction of the work, and is clothed in fiction only to 
make it easier to visualiza. 

Let us consider an "augmented" architect at work. He sits at a working sta- 
tion that has a visual display screen some three feet on a side; this is his working 
surface, and is controlled by a computer (his "clerk") with which he can com- 
municate by means of a small keyboard and various other devices. 
He is designing a building. He has already dreamed up several basic layouts 
and structural forms, and is trying them out on the screen. The surveying 
data 
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for the layout he is working on now have already been entered, 
and he has just coaxed the "clerk" to show him a perspective view 
of the steep hillside building site with the roadway above, symbolic 
representations of the various trees that are to remain on the lot, 
and the service tie points for Die different Utilities. The view 
occupies the left two-thirds of the screen. With a "pointer," he 
indicates two points of interest, moves his left hand rapidly over the 
keyboard, and the distance and elevation between the points 
indicated appear on the right-hand third of the screen. 

Now he enters a reference Une with his "pointer" and the 
keyboard. Gradu-ally the screen begins to show the work he is 
doing — a neat excavation appears in the hillside, revises itself 
slightly, and revises itself again. After a moment, the architect 
changes the scene on the screen to an overhead plan view of the 
site, still showing the excavation. A few minutes of study, and he 
enters on the keyboard a list of items, checking each one as it 
appears on the screen, to be studied later. 

Ignoring the representation on the display, the architect next 
begins to enter a series of specifications and data — a six-inch slab 
fioor, twelve-inch concrete walls eight feet high within the 
excavation, and so on. When he has finished, the revised scene 
appears on the screen. A structure is taking shape. He examines it, 
adjusts it, pauses long enough to ask for handbook or catalog 
information from the "clerk" at various points, and readjusts 
accordingly. He often recalls from the "clerk" his working lists of 
specifications and considerations to refer to them, modify them, or 
add to them. These lists grow into an ever-more-detailed, in- 
terlinked structure, which represents the maturing thought behind 
the actual design. 

Prescribing different planes here and there, curved surfaces 
occasionally, and moving the whole structure about five feet, he 
finally has the rough externa! form of the building balanced nicely 
with the setting and he is assured that this form is basically 
compatible with the materials to be used as well as with the 
function of the building. 



Now he begins to enter detailed information about the interior. 
Here the ca-pability of the "clerk" to show him any view he wants to 
examine (a slice of the interior, or how the structure would look 
from the roadway above) is important. He enters particular fixture 
designs, and examines them iri a particular room. He checks to 
make sure that sun glare from the windows will not blind a driver 
on the roadway, and the "clerk" computes the information that one 
window will re-flect strongly onto the 

roadway between 6 and 6:30 on midsummer mornings. 
Next he begins a functional analysis. He has a list of the people 
who will oc-cupy this building, and the daily sequences of their 
activities. The "clerk" allows 
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him to follow each in turn, examining how doors swing, where special lighting 
might be needed. Finally he has the "clerk" combine all of these sequences of ac- 
tivity to indicate spots where traffic is heavy in the building, or where congestion 
might occur, and to determine what the severest drain on the Utilities is likely to 
be. 

All of this information (the building design and its associated "thought struc- 
ture") can be stored on a tape to represent the "design manual" for the building. 
Loading this tape into his own "clerk," another architect, a builder, or the client 
can maneuver within diis "design manual" to pursue whatever details or insights 
are of interest to him — and can append special notes that are integrated into the 
"design manual" for his own or someone else's later benefit. 

In such a future working relationship between human problem-solver and 
computer "clerk," the capability of the computer for executing mathematical 
processes would be used whenever it was needed. However, the computer has 
many other capabilities for manipulating and displaying information that can be 
of significant benefit to the human in nonmathematical processes of planning, or- 
ganizing, studying, etc. Every person who does his thinking with symbolized 
concepts (whether in the form of the English language, pictographs, formal logic, 
or mathematics) should be able to benefit significandy. 



B. Objedive ofthe Study 

The objective of this study is to develop a conceptual framework within which 
could grow a coordinated research and development program whose goals would 
be the following: (1) to find the factors that limit the ehectiveness ofthe 
individual basic information-handling capabilities in meeting the various needs 
of so-ciety for problem solving in its most general sense; and (2) to develop 
new techm'ques, procedures, and systems that will better match these basic 
capabilities to the needs, problems, and progress of society. We have placed the 
following specifications on this framework: 

3. That itprovideperspective for both long-range basic research and researchthat 
will yield practical results soon. 
4. That it indicate what this augmentation will actually involve in the way of 

changes in working environment, in thinking, in skills, and in methods of 

working. 

That it be a basis for evaluating the possible relevance of work and knowl-edge 
from existing fields and for assimilating whatever is relevant. 
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4. That it reveal areas where research is possible and ways to assess the re- 
search, be a basis for choosing starting points, and indicate how to de- 
velop appropriate methodologies for the needed research. 

Two points need emphasis here. First, although a conceptual framework has 
been constructed, it is still rudimentary. Further search, and actual research, are 
needed for the evolution of the framework. Second, even if our conceptual 
framework did provide an accurate and complete basic analysis of the system 
from which stems a human's intellectual efFectiveness, the explicit nature of future 
im-proved systems would be highly affected by (expected) changes in our 
technol-ogy or in our understanding of the human being. 



II. conceptual framework 
A. General 

The conceptual framework we seek must orient us toward the real possibilities 
and problems associated with using modern technology to give direct aid to an 
individual in comprehending complex situations, isolating the significant fac-tors, 
and solving problems. To gain this orientation, we examine how individuals 
achieve their present level of efFectiveness, and expect that this examination will 
reveal possibilities for improvement. 

The entire effect of an individual on the world stems essentially from what he 
can transmit to the world through his limited motor channels. This in turn is 
based on inFormation received From the outside world through limited sensory 
channels; on information, drives, and needs generated within him; and on his 
processing of that information. His processing is of two kinds: that which he is 
generally conscious of (recognizing patterns, remembering, visualizing, ab- 
stracting, deducing, inducing, etc.), and that involving the unconscious 
processing and mediating of received and self-generated information, and the 
unconscious mediating of conscious processing itself. 

The individual does not use this information and this processing to grapple 
directly with the sort of complex situation in which we seek to give him help. He 
uses his innate capabilities in a rather more indirect fashion, since the situation 
is generally too complex to yield directly to his motor actions, and always too 
complex to yield comprehensions and solutions from direct sensory inspection 
and use of basic cognitive capabilities. For instance, an aborigine who possesses 
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all of our basic sensory-mental-motor capabilities, but does not possess our 
back-ground of indirect knowledge and procedure, cannot organiza the proper 
direct actions necessary to drive a car through traffic, request a book from the 
library, cali a committee meeting to discuss a tentative plan, cali someone on 
the tele-phone, or compose a letter on the typewriter. 

Our culture has evolved means for us to organize the little things we can 
do with our basic capabilities so that we can derive comprehension from 
truly complex situations, and accomplish the processes of deriving and 
imple-menting problem solutions. The ways in which human capabilities are 
thus extended are here called augmentation means, and we define four basic 
classes of them: 

10. Artifacts — physical objects designed to provide for human comfort, for 
themanipulation of things or materials, and for the manipulation of 
symbols. 

11. Language — the way in which the individual pareeis out the picture of 
hisworld into the concepts that his mind uses to model that world, and 
thesymbols that he attaches to those concepts and uses in consciously 
ma-nipulating the concepts ("thinking"). 

12. Methodology — the methods, procedures, strategies, etc., with which an 
individual organizes his goal-centered (problem-solving) activity. 

13. Training — the conditioning needed by the human being to bring his skills 
in using Means 1,2, and 3 to the point where they are operationally effec- 
tive. 

The system we want to improve can thus be visualized as a trained human 
being together with his artifacts, language, and methodology. The explicit new 
system we contemplate will involve as artifacts computers, and computer- 
controlled information-storage, information-handling, and information-display 
devices. The aspects of the conceptual framework that are discussed here 
are primarily those relating to the human being's ability to make significant use 
of such equip-ment in an integrated system. 

Pervading all of the augmentation means is a particular structure or 
organ-ization. While an untrained aborigine cannot drive a car through 
trafile, be-cause he cannot leap the gap between his cultural background 
and the kind of world that contains cars and trafEc, it is possible to move 
step by step through an organized training program that will enable him to drive 
effectively and safely. In other words, the human mind neither learns nor 
acts by large leaps, but by steps organized or structured so that each 
one depends upon previous 

stcpa. 
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Although the size of the step a human being can take in comprehension, in- 
novation, or execution is small in comparison to the over-all size of the step 
needed to solve a complex problem, human beings nevertheless do solve com- 
plex problems. It is the augmentation means that serve to break down a large 
problem in such a way that the human being can walk through it with his little 
steps, and it is the structure or organization of these little steps or actions that we 
discuss as process hierarchies. 

Every process of thought or action is made up of sub-processes. Let us con- 
sider such examples as making a pencil stroke, writing a letter of the alphabet, or 
making a plan. Quite a few discrete muscle movements are organized into the 
making of a pencil stroke; similarly, making particular pencil strokes and making a 
plan for a letter are complex processes in themselves that become sub-processes 
to the over-all writing oian alphabetic character. 

Although every sub-process is a process in its own right, in that it consists of 
further sub-processes, there seems to be no point here in looking for the ultimate 
"bottom" of the process-hierarchical structure. There seems to be no way of 
telling whether or not the apparent "bottoms" (processes that cannot be further 
subdivided) exist in the physical world or in the limitations of human under- 
standing. 

In any case, it is not necessary to begin from the "bottom" in discussing 
particular process hierarchies. No person uses a process that is completely 
unique every time he tackles something new. Instead, he begins from a group of 
basic sensory-mental-motor process capabilities, and adds to these certain of 
the process capabilities of his artifacts. There are only a finite number of such 
basic human and artifact capabilities from which to draw. Furthermore, even quite 
different higher-order processes may have in common relatively high-order sub- 
processes. 

When a man writes prose text (a reasonably high-order process), he makes 
use of many processes as sub-processes that are common to other high-order 
processes. For example, he makes use of planning, composing, dictating. The 
process of writing is utilized as a sub-process within many different processes of 
a still higher order, such as organizing a committee, changing a policy, and so on. 

What happens, then, is that each individual develops a certain repertoire of 
process capabilities from which he selects and adapts those that will compose the 
processes that he executes. This repertoire is like a tool kit, and just as the me- 
chanic must know what his tools can do and how to use them, so the intellectual 
worker must know the capabilities of his tools and have good methods, strategies, 
and rules of thumb for making use of them. All of the process capabilities in the 
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individual's repertoire rest ultimately upon basic capabilities within him or his 
artifacts, and the entire repertoire represents an inter-knit, hierarchical 
structure (which we often cali the repertoire hierarchy). 

We find three general categories of process capabilities within a typical 
individual's repertoire. There are those that are executed completely within 
the human integument, which we cali explicit-human process capabilities; 
there are those possessed by artifacts for executing processes without 
human interven-tion, which we cali explicit-artifact process capabilities; and 
there are what we cali the composite process capabilities, which are derived 
from hierarchies con-taining both of the other kinds. 

We assume that it is our H-LAM/T system (Human using Language, Artifacts, 
Methodology, in which he is Trained) that has the capability and that per-forms 
the process in any instance of use of this repertoire. Let us look within the 
process structure for the LAM/T ingredients, to get a better "feel" for our mod- 
els. Consider the process of writing an important memo. There is a particular 
concept associated with this process — that of putting information into a 
formal package and distributing it to a set of people for a certain kind of 
considera-tion — and the type of information package associated with this 
concept has been given the special name of memorandum. Already the system 
language shows the effect of this process — i.e., a concept and its name. 
The memo-writing process may be executed by using a set of process 
capabilities (in intermixed or repetitive form) such as the following: planning, 
devel-oping subject matter, composing text, producing hard copy, and 
distributing. There is a definite way in which these sub-processes will be 
organized that represents part of the system methodology. Each of these sub- 
processes represents a functional concept that must be a part of the system 
language if it is to be organized effectively into the human's way of doing 
things, and the symbolic por- 

t&ra} ofeach coxcqvt jnust be such Oiat the human can work with it and 
rememherit. 

If the memo is simple, a paragraph or so in length, then the first three 
processes may well be of the explicit-human type (i.e., it may be planned, 
devel-oped, and composed within the mind) and the last two of the composite 
type. If it is a complex memo, involving a good deal of careful planning and 
development, then all of the sub-processes might well be of the composite type 
(e.g., at least in-cluding die use of pendi and paper artifacts), and diere 
migrtbe many dtfferent applications of some of the process capabilities 
within the total process (i.e., 
successwe, 
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occasion and sequence, would indeed enable the execution of the memo- 
writing process. However, the very process of organizing and supervising the 
utilization of these sub-process capabilities is itself a most importan! sub- 
process of the memo-writing process. Henee, the sub-process capabilities as 
listed would not be complete without the addition of a seventh capability — what 
we cali the exec-utive capability. This is the capability stemming from habit, 
strategy, rules of thumb, prejudice, learned method, intuition, unconscious 
dictates, or combina-tions thereof, to cali upon the appropriate sub-process 
capabilities with a particular sequence and timing. An executive process 
(i.e., the exercise of an executive capability) involves such sub-processes as 
planning, selecting, and supervising, and it is really the executive processes 
that embody all of the method-ology in the H-LAM/T system. 

To illustrate the capability-hierarchy features of our conceptual framework, 
let us consider an artifact innovation appearing directly within the relatively low- 
order capability for composing and modifying written text, and see how this 
can affect a (or, for instance, your) hierarchy of capabilities. Suppose you had 
a new wridng machine — think of it as a high-speed electric typewriter with 
some spe-cial features. You could operate its keyboard to cause it to write text 
much as you could use a conventional typewriter. But the printing mechanism 
is more corn-plicated; besides printing a visible character at every stroke, it 
adds special en-coding features by means of invisible selective components in 
the ink and special shaping of the character. 

As an auxiliary device, there is a gadget that is held like a pencil and, 
instead of a point, has a special sensing mechanism that you can pass over a 
line of the special printing from your wridng machine (or one like it). The signais 
which this reading stylus sends through the flexible connecdng wire to the 
writing machine are used to determine which characters are being sensed 
and thus to cause the automatic typing of a duplicate string of characters. 
An information-storage mechanism in the writing machine permits you to 
sweep the reading stylus over the characters much faster than the writer can 
type; the writer will catch up with you when you stop to think about what word 
or string of words should be du-plicated next, or while you reposition die 
straight-edge guide along which you run the stylus. 

This writing machine would permit you to use a new process of composing 
text. For instance, trial drafts could rapidly be composed from re-arranged ex- 
cerpts of oid drafts, togedier with new words or passages which you stop to 
type in. Your first draft could represent a free outpouring of thoughts in any 



order, with the inspection of foregoing thoughts continuously stimulating new 
considera- 
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tions and ideas to be entered. If the tangle of thoughts represented by the 
draft became too complex, you would compile a reordered draft quickly. It 
would be practical for you to accommodate more complexity in the trails of 
thought you might build in search of the path that suits your needs. 

You can integrate your new ideas more easily, and thus harness your 
creativ-ity more continuously, if you can quickly and flexibly change your 
working record. If it is easier to update any part of your working record to 
accommodate new developments in thought or circumstance, you will find it 
easier to incorporate more complex procedures in your way of doing things. 
This will proba-bly allow you to accommodate the extra burden associated 
with, for instance, keeping and using special files whose contents are both 
contributed to and uti-lized by any current work in a flexible manner — which in 
turn enables you to devise and use even-more-complex procedures to better 
harness your talents in your particular working situation. 

The important thing to appreciate here is that a direct new innovation in one 
particular capability can have far-reaching effects throughout the rest of your 
capability hierarchy. A change can propagate up through the capability 
hierarchy; higher-order capabilities that can utilize the initially changed 
capability can now reorganize to take special advantage of this change and 
of the intermediate higher-capability changes. A change can propagate down 
through the hierarchy as a result of new capabilities at the high level and 
modification possibilities la-tent in lower leveis. These latent capabilities may 
previously have been unusable in the hierarchy and become usable because 
of the new capability at the higher level. 

The writing machine and its flexible copying capability would occupy you for 
a long time if you tried to exhaust the reverberating chain of associated 
possibilities for making useful innovations within your capability hierarchy. This 
one innovation could trigger a rather extensive redesign of this hierarchy; your 
way of accomplishing many of your tasks would change considerably. 
Indeed, this process characterizes the sort of evolution that our intellect- 
augmentation means have been undergoing since the first human brain 
appeared. 

To our objective of deriving orientation about possibilities for actively pur- 
suing an increase in human intellectual effectiveness, it is important to realize 
that we must be prepared to pursue such new-possibility chains throughout 
the en-tire capability hierarchy (calling for a "system" approach). It is also 
important te realize that we must be oriented to the synthesis of new 



capabilities from reorga-nization of other capabilities, both oid and new, that 
exist throughout the hierarchy (calling for a "system-engineering" approach). 
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Individuals who operate effectively in our culture have already been 
considerably "augmented." Basic human capabilities ibr sensing stimuli, 
performing numer-ous mental operations, and for communicating with the 
outside world, are put to work in our society within a system — an H-LAM/T 
system — the individual augmented by the language, artifacts, and 
methodology in which he is trained. Fur-thermore, we suspect that improving 
the effectiveness of the individual as he operales in our society should be 
approached as a system-engineering prob-lem — that is, the H-LAM/T 
system should be studied as an interacting whole from a synthesis-oriented 
approach. 

This view of the system as an interacting whole is strongly bolstered by con- 
sidering the repertoire hierarchy of process capabilities that is structured from 
the basic ingredients within the H-LAM/T system. The realization that any 
potencial change in language, artifact, or methodology has importance only 
relative to its use within a process, and that a new process capability 
appearing anywhere within that hierarchy can make practical a new 
consideration of latent change possibilities in many other parts of the 
hierarchy — possibilities in either language, artifacts, or methodology — brings 
out the strong interrelationship of these three augmentation means. 

Increasing the effectiveness of the individual's use of his basic capabilities 
is a problem in redesigning the changeable parts of a system. The system is 
actively engaged in the continuous processes (among others) of developing 
comprehen-sion within the individual and of solving problems; both processes 
are subject to human motivation, purpose, and will. To redesign the system's 
capability for performing these processes means redesigning all or part of the 
repertoire hierarchy. To redesign a structure, we must learn as much as we can 
of what is known about the basic materials and components as they are 
utilized within the structure; beyond that, we must learn how to view, to 
measure, to analyze, and to evaluate in terms of the functional whole and its 
purpose. In this particular case, no existing analytic theory is by itself 
adequate for the purpose of analyzing and evaluating over-all system 
performance; pursuit oian improved system thus de-mands the use oi 
experimental methods. 

It need not be just the very sophisticated or formal process capabilities 
that are added or modified in this redesign. Essentially any of the processes 
utilized by a representative human today — the processes that he thinks of 
when he look! ahead to his day's work — are composite processes of the sort 



that involve exter nal composing and manipulating of symbols (text, sketches, 
diagrams, lists, etc.) 
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Many of the external composing and manipulating (modifying, rearranging) 
processes serve such characteristically "human" activities as playing with 
forms and relationships to see what develops, cut-and-try multiple-pass 
development of an idea, or listing Items to reflect on and then rearranging and 
extending them as thoughts develop. 

Existing, or near-future, technology could certainly provide our professional 
problem-solvers with the artifacts they need to have for duplicating and 
rearranging text before their eyes, quickly and with a minimum of human 
effort. Even so apparently minor an advance could yield total changes in an 
individual's repertoire hierarchy that would represent a great increase in over-all 
effectiveness. Normally the necessary equipment would enter the market 
slowly; changes from the expected would be small, people would change their 
ways of doing things a little at a time, and only gradually would their 
accumulated changes create mar-kets for more radical versions of the 
equipment. Such an evolutionary process has been typical of the way our 
repertoire hierarchies have grown and formed. 

But an active research effort, aimed at exploring and evaluating possible in- 
tegrated changes throughout the repertoire hierarchy, could greatly accelerate 
this evolutionary process. The research effort could guide the product 
development of new artifacts toward taking long-range meaningful steps; 
simultaneously, com-petitively minded individuals who would respond to 
demonstrated methods for achieving greater personal effectiveness would 
create a market for the more radical equipment innovations. The guided 
evolutionary process could be expected to be considerably more rapid than 
the traditional one. 

The category of "more radical innovations" includes the digital computer as 
a tool for the personal use of an individual. Here there is not only promise of 
great flexibility in the composing and rearranging of text and diagrams before 
the individual's eyes, but also promise of many other process capabilities that 
can be integrated into the H-LAM/T system's repertoire hierarchy. 



C. Detailed Discussion ofthe H-LAM/T System*!, the source of intelligence 
When one looks at a computer system that is doing a very complex job, he 
sees on the surface a machine that can exe-cute some extremely 



sophisticated processes. If he is a layman, his concept oi what provides this 
sophisticated capability may endow the machine with a mys-terious power to 
sweep information through perceptive and intelligent synthetic thinking 
devices. Actually this sophisticated capability results from a very clevei 
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arganizational hierarchy so that pursuit of the source of intelligence within this 
system would take one down through layers of functional and physical 
organization that become successively more primitive. 

To be more specific, we can begin at the top and list the major leveis down 
through which we would pass if we successively decomposed the functional 
el-ements of each level, in search of the "source of intelligence." A 
programmer could take us down through perhaps three leveis (depending 
upon the sophisti-cation of the total process being executed by the computer) 
perhaps depicting the organization at each level with a flow chart. The first level 
down would organize functions corresponding to statements in a problem- 
oriented language (e.g.. ALGOL or COBOL), to achieve the desired over-all 
process. The second level down would organize lesser functions into the 
processes represented by first-level statements. The third level would perhaps 
show how the basic machine com-mands (or rather the processes which they 
represent) were organized to achieve each of the functions of the second 
level. 

Then a machine designer could take over, and with a block diagram of the 
computer's organization he could show us (Level 4) how the different 
hardware units (e.g., random-access storage, arithmetic registers, adder, 
arithmetic control) are organized to provide the capability of executing 
sequences of the commands used in Level 3. The logic designer could then 
give us a tour of Level 5, also using block diagrams, to show us how such 
hardware elements as pulse gates, flip-flops, and AND, OR, and NOT circuits 
can be organized into networks giving the functions utilized at Level 4. For 
Level 6 a circuit engineer could show us diagrams revealing how components 
such as transistors, resistors, capacitors, and diodes can be organized into 
modular networks that provide the functions needed for the elements of 
Level 5. 

Device engineers and physicists of different kinds could take us down 
through more layers. But rather soon we have crossed the boundary 
between what is man-organized and what is nature-organized, and are 
ultimately dis-cussing the way in which a given physical phenomenon is 
derived from the in-trinsic organization of sub-atomic particles, with our ability to 
explain succeeding layers blocked by the exhaustion of our present human 
comprehension. 



If we then ask ourselves where that intelligence is embodied, we are forced to 
concede that it is elusively distributed throughout a hierarchy of functional 
processes — a hierarchy whose foundation extends down into natural 
processes below the depth of our comprehension. If there is any one thing 
upon which this "intelligence" depends, it would seem to be organization, 
The biologists and ohvsiolodsts use a term "synergism" to designate (from 
Webster's Unabridged 
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Dictionary, Second Edition) the "... cooperative action of discrete agencies 
such that the total effect is greater than the sum of the two effects taken 
indepen-dendy..." This tenn seems directly applicable here, where we could 
say that syn-ergism is our most likely candidate for representing the actual 
source of intelligence. 

Actually, each of the social, life, or physical phenomena we observe about 
us would seem to derive from a supporting hierarchy of organized functions 
(or processes), in which the synergistic principle gives increased 
phenomenological sophistication to each succeedingly higher level of 
organization. In particular, the intelligence of a human being, derived ultimately 
from the characteristics of individual nerve cells, undoubtedly results from 
synergism. 

2. intelligence amplification It has been jokingly suggested several times 
during the course of this study that what we are seeking is an "intelligence 
amplifier." (The term is attributed originally to W. Ross Ashby. 2 At first this temi 
was rejected on the grounds that in our view one's only hope was to make a 
bet-ter match between existing human intelligence and the problems to be 
tackled, rather than in making man more intelligent. But deriving the concepts 
brought out in the preceding section has shown us that indeed this term 
does seem applicable to our objective.) 

Accepting the term "intelligence amplification" does not imply any attempt to 
increase native human intelligence. The term "intelligence amplification" 
seems applicable to our goal of augmenting the human intellect in that the 
entity to be produced will exhibit more of what can be called intelligence than 
an unaided human could; we will have amplified the intelligence of the human 
by organizing his intellectual capabilities into higher leveis of synergistic 
structuring. What pos-sesses the amplified intelligence is the resulting H- 
LAM/T system, in which the LAM/T augmentation means represent the 
amplifier of die human's intelligence. 



In amplifying our intelligence, we are applying the principle of synergistic 
structuring that was followed by natural evolution in developing the basic 
human :apabilities. What we have done in the development of our 
augmentation means s to construct a superstructure that is a synthetic 
extension of the natural structure upon which it is built. In a very real sense, 
as represented by the steady evo-ution of our augmentation means, the 
development of "artificial intelligence" has )een going on for centuries. 

5. Two-DoMAiN system The human and the artifacts are the only physical 
•omnonents in the H-LAM/T svstem. It is upon their capabilities that the ulti- 
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mate capability of the system will depend. This was implied in the earlier state- 
ment that every composite process of the system decomposes ultimately inte 
explicit-human and explicit-artifact processes. There are thus two separate 
domains of activity within the H-LAM/T system: that represented by the 
human, in which all explicit-human processes occur; and that represented by 
the artifacts in which all explicit-artifact processes occur. In any composite 
process, there is cooperative interaction between the two domains, requiring 
interchange of en-ergy (much of it for information exchange purposes only). 
Figure 1 depicts this two domain concept and embodies other concepts 
discussed below. 



Outside World 

H-LAM/T System 

Matching^Processes 
IVTafeWig Processes 



Human Processes 
Man-Artifact-Interface 
Artifact Processes 

FIGURE 1 . Portrayal ofthe TinoActove Domains within the H-LMA/TSystem. 

Where a complex machine represents the principal artifact with which a 
Human being cooperates, the term "man-machine interface" has been used 
for some years to represent the boundary across which energy is exchanged 



between the two domains. However, the "man-artifact interface" has existed for 
centuries, sver since humans began using artifacts and executing composite 
processes. 

Exchange across this "interface" occurs when an explicit-human process is 
coupled to an explicit-artifact process. Quite often these coupled processes 
are designed for just this exchange puipose, to provide a functional match 
between other explicit-human and explicit-artifact processes buried within 
their respective domains that do the more significant things. For instance, the 
finger and hand motions (explicit-human processes) activate key-linkage 
motions in the typewriter (couple to explicit-artifact processes). But these are 
only part ofthe matching processes between the deeper human processes 
that direct a given word to be typed and the deeper artifact processes that 
actually imprint the ink marks on the naner. 
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The outside world interacts with our H-LAM/T system by the exchange of 
snergy with either the individual or his artifact. Again, special processes are 
often designed to accommodate this exchange. However, the direct concern 
of our present study lies within the system, with the internal processes that are 
and can be significantly involved in the effectiveness of the system in 
developing the human's comprehension and pursuing the human's goals.... 

5. capability repertoire hierarchy The concept of our H-LAM/T system 
possessing a repertoire of capabilities that is structured in the form of a 
lierarchy is most useful in our study. We shall use it in the following to tie to- 
rether a number of considerations and concepts. 

There are two points of focus in considering the design of new repertoire 
hierarchies: the materials with which we have to work, and the principles by 
which lew capability is constructed from these basic materials. 

i. Basic Capabilities 

Materials in this context are those capabilities in the human and in the artifact 
do-nains from which all other capabilities in the repertoire hierarchy must be 
con-jtructed. Each such basic capability represents a type of functional 
component vith which the system can be built, and a thorough Job of 
redesigning the system :alls for making an inventory of the basic capabilities 
available. Because we are -.xploring for perspective, and not yet 
recommending research activities, we are ree to discuss and define in more 
detail what we mean by "basic capability" vithout regard to the amount of 
research involved in making an actual inventory. The two domains, human and 
artifact, can be explored separately for their >asic capabilities. In each we 
can isolate two classes of basic capability; these lasses are distinguished 



according to whether or not the capability has been >ut to use within out 
augmentation means. The first class (those in use) can be ound in a 
methodical manner by analyzing present capability hierarchies. For ex-mple, 
select a given capability at any level in the hierarchy and ask yourself if it an be 
usefully changed by any means that can be given consideration in the 
ugmentation research contemplated. If it can, then it is not basic but it can be 
iecomposed into an eventual set of basic capabilities. As you proceed down 
lirough the hierarchy you will begin to encounter capabilities that cannot be use- 
ally changed, and these will make up your inventory of basic capabilities. Ulti- 
lately, every such recursive decomposition of a given capability in the hierarchy 
rill find everv one of its branchine paths terminated by basic capabilities. Be- 



82 Uouglas tngemarr 

such decomposition search with different capabilities in the hierarchy 
will eventually uncover all of those basic capabilities used within that hierarchy 
or augmentation system. Many of the branching paths in the decomposition 
of a given higher-order capability will terminate in the same basic capability, 
since a given basic capability will often be used within many different higher- 
order capabilities. 

Determining the class of basic capabilities not already utilized within exist- 
ing augmentation systems requires a different exploration method. Examples 
of this method occur in technological research, where analytically oriented 
re-searchers search for new understandings of phenomena that can add to 
the research engineer's list of things to be used in the synthesis of better 
artifacts. 

Before this inventorying task can be pursued in any specific instance, 
some criteria must be established as to what possible changes within the H- 
LAM/T system can be given serious consideration. For instance, some 
research situa-tions might have to disallow changes which require extensive 
retraining, or which require undignified behavior by the human. Other 
situations might admit changes requiring years of special training, very 
expensive equipment, or the use ofspecialdrugs. 

The capability for performing a certain finger action, for example, may not be 
basic in our sense of the word. Being able to extend the finger a certain 
distance would be basic, but the strength and speed of a particular finger 
motion and its coordination with higher actions generally are usefully 
changeable and there-fore do not represent basic capabilities. What would be 
basic in this case would perhaps be the processes whereby strength could be 



increased and coordinated movement patterns learned, as well as the basic 
movement range established by the mechanical-limit loci of the muscle- 
tendon-bone system. Similar capability breakdowns will occur for sensory and 
cognitive capabilities. 

b. Structure Types 

1) General The fundamental principle used in building sophisticated 
capabilities from the basic capabilities is structuring — the special type of 
structuring (which we have termed synergetic) in which the organization of a 
group of ele-ments produces an effect greater than the mere addition of their 
individual effects, Perhaps "purposeful" structuring (or organization) would 
serve us as well, bul since we aren't sure yet how the structuring concept 
must mature for our needs we shall tentatively stick with the special modifier, 
"synergetic." We are develenme a Erowine: awareness of the significan! and 
pervasive nature of such struc- 
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ture within every physical and conceptual thing we inspect, where the hierar- 
chical form seems almost universally present as stemming from successive 
leveis of such organizacion. 

The fundamental entities that are being structured in each and every case 
seems to be what we could cali processes, where the most basic of 
physical processes (involving fields, charges, and momenta associated with 
the dynamics of fundamental particles) appear to be the hierarchical base. 
There are dynamic electro-optical-mechanical processes associated with the 
function of our arti-facts, as well as metabolic, sensory, motor, and cognitive 
processes of the human, which we find to be relatively fundamental 
components within the structure of our H-LAM/T system — and each of these 
seems truly to be ultimately based (to our degree of understanding) upon the 
above mentioned basic physical processes. The elements that are 
organized to give fixed structural form to our physical objects — e.g., the 
"element" of tensile strength of a material — are also de-rived from what we 
could cali synergetic structuring of the most basic physical processes. 

But at the level of the capability hierarchy where we wish to work, it seems 
useful to us to distinguish several different types of structuring — even though 
each type is fundamentally a structuring of the basic physical processes. 
Tenta-tively we have isolated five such types — although we are not sure how 
many we shall ultimately want to use in considering the problem of augmenting 



the human intellect, nor how we might divide and subdivide these different 
manifestations of physical-process structuring. We use the terms "mental 
structuring," "concept structuring," "symbol structuring," "process structuring," 
and "physical structuring." 

2) Mental Structuring Mental structuring is what we cali the internal organi- 
zation of conscious and unconscious mental images, associations, or concepts 
(or whatever it is that is organized within the human mind) that somehow 
manages to provide the human with understanding and the basis for such as 
judgment, in-tuition, inference, and meaningful action with respect to his 
environment. There is a term used in psychology, "cognitive structure," which 
so far seems to represen! just what we want for our concept of mental 
structure, but we will not adopt t until we become more sure of what the 
accepted psychological meaning is and jf what we want for our conceptual 
framework. 

For our present purpose, it is irrelevant to worry over what the fundamen- 
al mental "things" being structured are, or what mechanisms are 
accomplish-ng the structurine: or makine: use of what has been 
structured. We fppl 
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reasonably safe in assuming that learning involves some kind of meaningful 
or-ganization within the brain, and that whatever is so organized or structured 
rep-resents the operating model of the individual's universe to the mental 
mechanisms that derive his behavior. And further, our assumption is that 
when the human in our H-LAM/T system makes the key decision or action 
that leads to the solution of a complex problem, it will stem from the state of 
his mental structure at that time. In this view, then, the basic purpose of the 
sys-tem's activity on that problem up to that point has been to develop his 
mental structure to the state from which the mental mechanisms could derive 
the key action. 

Our school systems attest that there are specific experiences that can be 
given to a human that will result in development of his mental structure to the 
point where the behavior derived therefrom by his mental mechanisms shows 
us that he has gained new comprehension — in other words, we can do a 
certain amount from outside the human toward developing his mental 
structure. Independent students and researchers also attest that internally 
directed behavior on the part of an individual can directly aid his structure- 
building process. 

We don't know whether a mental structure is developed in a manner 
analo-gous to (a) development of a garden, where one provides a good 
environment, plants the seeds, keeps competing weeds and injurious pests 



out, but otherwise has to let natural processes take their course, or to (b) 
development of a basket-ball team, where much exercise of skills, patterns, 
and strategies must be provided so that natural processes can slowly knit 
together an integration, or to (c) development of a machine, where carefully 
formed elements are assembled in a precise, planned manner so that natural 
phenomena can immediately yield planned function. We don't know the 
processes, but we can and have developed empiri-cal relationships between 
the experiences given a human and the associated man-ifestations of 
developing comprehension and capability, and we see the near-future 
course of the research toward augmenting the human's intellect as de-pending 
entirely upon empirical findings (past and future) for the development of 
better means to serve the development and use of mental structuring in the 
human. 

We don't mean to imply by this that we renounce theories of mental 
processes. What we mean to emphasize is that pursuit of our objective need 
nol wait upon the understanding of the mental processes that accomplish 
(what w£ cali) mental structuring and that derive behavior therefrom. It would 
be to ignor< the emphases of our own conceptual framework not to make fullest 
use of any the nrv that nrovided a working; explanation for a group of 
empirical data. What'i 
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more, our entire conceptual framework represents the first pass at a 
"theoreticaP model with which to organize our thinking and action. 

3) Concept Structuring Within our framework we have developed the working 
assumption that the manner in which we seem to be able to provide 
experiences that favor the development of our mental structures is based upon 
concepts as a "medium of exchange." We view a concept as a tool that can 
be grasped and used by the mental mechanisms, that can be composed, 
interpreted, and used by the natural mental substances and processes. The 
grasping and handling done by these mechanisms can often be facilitated if 
the concept is given an explicit "handle" in the form of a representative 
symbol. Somehow the mental mechanisms can learn to manipulate images 
(or something) of symbols in a meaning-ful way and remain calmly confident 
that the associated conceptual manipulations are within cali. 

Concepts seem to be structurable, in that a new concept can be composed 
of an organization of established concepts. For present purposes, we can 



view a concept structure as something which we might try to develop on 
paper for our-selves or work with by conscious thought processes, or as 
something which we try to communicate to one another in serious discussion. 
We assume that, for a jfiven unit of comprehension to be imparted, there is a 
concept structure (which :an be consciously developed and displayed) that 
can be presented to an individual in such a way that it is mapped into a 
corresponding mental structure tvhich provides the basis for that individual's 
"comprehending" behavior. Our >vorking assumption also considers that 
some concept structures would be bet-:er for this purpose than others, in that 
they would be more easily mapped by the ndividual into workable mental 
structures, or in that the resulting mental structures enable a higher degree 
of comprehension and better solutions to prob-ems, or both. 

A concept structure often grows as part of a cultural evolution — either on a 
arge scale within a large segment of society, or on a small scale within the 
activ-ty domain of an individual. But it is also something that can be directly 
designed ir modified, and a basic hypothesis of our study is that better 
concept structures an be developed — structures that when mapped into a 
human's mental structure ^11 significantly improve his capability to 
comprehend and to find solutions /ithin his complex-problem situations. 

A natural language provides its user with a ready-made structure of 
concepts lat establishes a basic mental structure, and that allows relatively 
flexible, eneral-purpose concept structuring. Our concept of "lansoiase" as 
one of the 
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basic means for augmenting the human intellect embraces all of the concept 
structuring which the human may make use of. 

4) Symbol Structuring The other important part of our "language" is the wav in 
which concepts are represented — the symbols and symbol structures. Words 
structured into phrases, sentences, paragraphs, monographs — charts, lists, dia- 
grams, tables, etc. A given structure of concepts can be represented by any of ai. 
infinite number of different symbol structures, some of which would be much 
better than others for enabling the human perceptual and cognitive apparatus k 
search out and comprehend the conceptual matter of significance and/or inter- 
est to the human. For instance, a concept structure involving many numerical dau 
would generally be much better represented with Arabic rather than Roman 



numerais and quite likely a graphic structure would be better than a tabular 
structure. 

But it is not only the form of a symbol structure that is important. A probleir 
solver is involved in a stream of conceptual activity whose course serves his 
mental needs of the moment. The sequence and nature of these needs are quite 
variable, and yet for each need he may benefit significantly from a form of 
symrx'. structuring that is uniquely efficient for that need. 

Therefore, besides the forms of symbol structures that can be constructs 
and portrayed, we are very much concerned with the speed and flexibility with 
which one form can be transformed into another, and with which new mater. A 
can be located and portrayed. 

We are generally used to thinking of our symbol structures as a pattern 
marks on a sheet of paper. When we want a different symbol-structure view, we 
thitik of shifting our point of attention on the sheet, or moving a new sheet into 
position. But another kind of view might be obtained by extracting and orden:.; all 
statements in the local text that bear upon Consideration A of the argument-or by 
replacing all occurrences of specified esoteric words by one's own def tions. 
This sort of "view generation" becomes quite feasible with computer-controlled 
display system, and represents a very significan! capabi to build upon. 
With a computer manipulating our symbols and generating their portray; to us on 
a display, we no longer need think of our looking at the symbol struc which is 
stored — as we think of looking at the symbol structures stored in noi books, 
memos, and books. What the computer actually stores need be none our 
concern, assuming that it can portray symbol structures to us that aie sistent 
with the form in which we think our information is structured. 
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A given concept structure can be represented with a symbol structure that is 
:ompletely compatible with the computer's interna! way of handling symbols, 
ivith al! sorts of characteristics and relationships given explicit identifications 
that the user may never directly see. In fact, this structuring has immensely 
greater potential for accurately mapping a complex concept structure than 
does a structure an individual would find it practical to construct or use on 
paper. 

The computer can transform back and forth between the two-dimensional 
portrayal on the screen, of some limited view of the total structure, and the 
aspect of the n-dimensional interna! image that represents this "view." If the 
human adds to or modifies such a "view," the computer integrales the 
change into the internal-image symbol structure (in terms of the computer's 
favored symbols and structuring) and thereby automatically detects a certain 
proportion of his possible conceptual inconsistencies. 



Thus, inside ihis instrument (the computer) there is an internal-image, 
computer-symbol structure whose convolutions and multi-dimensionality we 
can learn to shape to represent to hitherto unattainable accuracy the 
concept structure we might be building or working with. This internal 
structure may have a form that is nearly incomprehensible to the direct 
inspection of a human (except in minute chunks). 

But let the human specify to the instrument his particular conceptual need oi 
:he moment, relative to this internal image. Without disrupting its own internal 
"eference structure in the slightest, the computer will effectively stretch, bend, 
bid, extract, and cut as it may need in order to assemble an internal 
substructure :hat is its response, structured in its own internal way. With the 
set of standard ranslation rules appropriate to the situation, it portrays to the 
human via its dis-)lay a symbol structure designed for his quick and accurate 
perception and comprehension of the conceptual matter pertinent to this 
internally composed ubstructure. 

No longer does the human work on stiff and limited symbol structures, where 
nuch of the conceptual conteni can only be implicitly designated in an indirect 
nd distributed fashion. These new ways of working are basically available with 
oday's technology — we have but to free ourselves from some of our limiting 
iews and begin experimenting with compatible sets of structure forras and 
irocesses for human concepts, human symbols, and machine symbols. 
) Process Structuring Essentially everything that goes on within the H-LAM/T 
ystem and that is of direct interest here involves the manipulation of concept 
and vmhol stmrtures in servire to the mental structure. Therefore, the 
orocesses 
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within the H-LAM/T system that we are most interested in developing are 
those that provide for the manipulation of all three types of structure. This 
brings us to the fourth category of structuring process structuring. 

As we are currently using it, the term process structuring includes the orga- 
nization, study, modification, and execution of processes and process 
structures. Whereas concept structuring and symbol structuring together 
represent the lan-guage component of our augmentation means, process 
structuring represents the methodology component (plus a little more, 
actually). There has been enough previous discussion of process structures 
that we need not describe the notion here, beyond perhaps an example or 
two. The individual processes (or actions) of my hands and fingers have to be 



cooperatively organized if the typewriter is to do my bidding. My successive 
actions throughout my working day are meant to cooperate toward a certain 
over-all professional goal. 

Many of the process structures are applied to the task of organizing, 
executing, supervising, and evaluating other process structures. Many of them 
are applied to the formation and manipulation of symbol structures (the 
purpose of which will often be to support the conceptual labor involved in 
process structuring). 

6) Physical Structuring Physical structuring, the last of the five types which we 
currently use in our conceptual framework, is nearly self-explanatory. It pretty 
well represents the artifact component of our augmentation means, insofar 
as their actual physical construction is concerned.... 

e. Flexibility in the Executive Role 

[T]here is finite human capability which must be divided between executive and 
direct-contributive activities [in the H-LAM/T system]. An important aspect of 
the multi-role activity of the human in the system is the development and 
manipulation of the symbol structures associated with both his direct- 
contributive roles and his executive roles. 

When the system encounters a complex situation in which comprehen- 
sion and problem solutions are being pursued, the direct-contributive roles 
re-quire the development of symbol structures that portray the concepts 
involved within the situation. But executive roles in a complex problem 
situation also re-quire conceptual activity — e.g., comprehension, selection, 
supervision — that can benefit from well-designed symbol structures and fast, 
flexible means for manip-ulating and displaying them. For complex processes, 
the executive problem posed to the human fof eaining the necessary 
comprehension and making a good plan) 
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may be tougher than the problem he faced in the role of direct-contributive 
worker. If the flexibility desired for the process hierarchies (to make room for 
human cut-and-try methods) is not to be degraded or abandoned, the executive 
activity wiH have to be provided with fast and flexible symbol-structuring 
techniques. 

The means available to humans today for developing and manipulating 
these symbol structures are both laborious and inflexible. It is hard enough to 
develop an initial structure of diagrams and text, but the amount of effort 
required to make changes is often prohibitively great; one settles for 
inflexibility. Also, the kind of generous flexibility that would be truly helpful 



calis for added symbol structuring just to keep track of the triais, branches, 
and reasoning thereto that are involved in the development of the subject 
structure; our present symbol-manipulation means would very soon bog 
down completely among the com-plexities that are involved in being more 
than just a little bit flexible. 

We find that the humans in our H-LAM/T systems are essentially working 
continuously within a symbol structure of some sort, shifting their attention from 
one structure to another as they guide and execute the processes that 
ultimately provide them with the comprehension and the problem solutions 
that they seek. This view increases our respect for the essential importance 
of the basic capa-bility of composing and modifying efEcient symbol 
structures. Such a capability depends heavily upon the particular concepts 
that are isolated and manipulated as entities, upon the symbology used to 
represent them, upon the artifacts that help to manipulate and display the 
symbols, and upon the methodology for developing and using symbol 
structures. In other words, this capability depends heavily upon proper 
language, artifacts, and methodology, our basic augmenta-tion means. 

When the course of action must respond to new comprehension, new in- 
sights and new intuitive flashes of possible explanations or solutions, it will not 
be an orderly process. Existing means of composing and working with 
symbolstructures penalize disorderly processes very heavily, and it is part of 
the realpromise in the automated H-LAM/T systems of tomorrow that the 
human canhave the freedom and power of disorderly processes 

VI. CONCLUSIONS 

Three principal conclusions may be drawn concerning the significance and 
im-plications of the ideas that have been presented. 

First, any possibility for improving the effective utilization of the intellectual 
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power of society's problem solvers warrants the most serious consideration. 
This is because man's problem-solving capability represents possibly the 
most important resource possessed by a society. The other contenders for 
first impor-tance are all critically dependen! for their development and use 
upon this resource. Any possibility for evolving an art or science that can 
couple directly and significantly to the continued development of that resource 
should warrant doubly serious consideration. 



Second, the ideas presented are to be considered in both of the above 
senses: the direct-development sense and the "art of development" sense. To 
be sure, the possibilities have long-term implications, but their pursuit and 
initial rewards await us now. By our view, we do not have to wait until we learn 
how the human mental processes work, we do not have to wait until we learn 
how to make com-puters more intelligent or bigger or faster; we can begin 
developing powerful and economically feasible augmentation systems on the 
basis of what we now know and have. Pursuit of further basic knowledge and 
improved machines will continue into the unlimited future, and will want to be 
integrated into the "art" and its improved augmentation systems — but getting 
started now will provide not only orientation and stimulation for these pursuits, 
but will give us improved problem-solving efFectiveness with which to carry 
out the pursuits. 

Third, it becomes increasingly clear that there should be action now — the 
sooner the better — action in a number of research communities and on an 
ag-gressive scale. We offer a conceptual framework and a plan for action, and 
we rec-ommend that these be considered carefully as a basis for action. If 
they be considered but found unacceptable, then at least serious and 
continued effort should be made toward developing a more acceptable 
conceptual framework within which to view the overall approach, toward 
developing a more acceptable plan of action, or both. 

This is an open plea to researchers and to those who ultimately motivate, fi- 
nance, or direct them, to turn serious attention toward the possibility of 
evolving a dynamic discipline that can treat the problem of improving 
intellectual efFectiveness in a total sense. This discipline should aim at 
producing a continuous cycle of improvements — increased understanding of 
the problem, improved means for developing new augmentation systems, 
and improved augmentation systems that can serve the world's problem 
solvers in general and this discipline'* workers in particular. After all, we spend 
great sums for disciplines aimed at understanding and harnessing nuclear 
power. Why not consider developing a discipline aimed at understanding and 
harnessing "neural power"? In the long run. the oower of the human intellect is 
reallv much the more important of the two. 
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"Diary: Audience 1966," A Year From A/oncfay (1966) 



John Cage. Variations V. Photo 
byfferve Gloaguen. Courtesy 
07 Merge Cunningham Vance 
Foundation. 



"What'll art become? A family reunion? Ifso, let's have jt 
with people in the round, each individual free to lend his 
attention whereverhe will. " 
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« As a musician, composer, artist, poet, and philosopher, John Cage's work 
rarely fit within the traditional boundaries of artistic practice. Originally from 
Los Angeles, where he was a student of the Viennese composer Arnold Schoen- 
berg, Cage's interdisciplinary interests led him far from the conventions of his 
musical training. In the late 1940s, during a residency at Black Mountain Col- 
lege, he developed his provocative "theater of mixed-means" in collaborations 
with the artists Robert Rauschenberg and Jasper Johns and the choreographer 
Merce Cunningham. These experiments gave birth to an explosion of 
performance art in the 1950s and 1960s that introduced al I types of actions, 
artifacts, noises, images, and movement into the performance space, which 
culminated in the Happenings. 

Cage embraced indeterminacy as an integral part of his process of composi- 
tion; this technique led him to include the participation of the audience in the cre- 
ation of his work. Inspired by Zen Buddhism, he reveled in an anarchy that 
dethrones the artist as the heroic, all-powerful arbiter of creative expression. He 
proposed instead a shift to an inclusive, participatory art that encourages inter- 
action between artist, performer and audience, one in which the latter "can sit 
quietly or make noises ... whisper, talk and even shout/This is best illustrated 
by his infamous piece from 1953, 4'33", in which the pianist David Tudor re- 
mained silent at his instrument for the prescribed duration of time. While the 
piece was initially met by critics as a scandal, Cage's intent was to encourage the 
listener to contemplate the passing of time freely while listening attentively to the 
random sounds of the concert hall. 

In this essay, Cage links the notion of an interactive listener to the concept of 
the computer as an agent of participation rather than as a servile, "labor- 
saving" device. His insightful analysis of the changing relationship between the 
artwork and the viewer has set the stage for much that has since become wide- 
spread in human-computer interaction, and has profoundly influenced later gen- 
erations of media artists exploring interactive strategies. » 

I. Are we an audience for computer art? The answer's not No; it's Yes. What 
we need is a computer that isn't labor-saving but which increases the work for 
us to do, that puns (this is McLuhan's idea) as well as Joyce revealing bridges 
(this is Brown's idea) where we thought there weren't any, turns us (my idea) 
not "on" but into artists. Orthodox seatins; arransement in synasvs A ies. Indians 
have known 
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it for ages: life's a dance, a play, illusion. Lila. Maya. Twentieth-century art's 
opened our eyes. Now music's opened our ears. Theatre? Just notice 
what's iround. (If what you want in India is an audience, Gita Sarabhai told 
me, all you need is one or two people.) II. He said: Listening to your music I 
find it provokes me. What should I do to enjoy it? Answer: There're many ways 
to help you. I'd give you a lift, for instance, if you were going in my direction, 
but the last thing I'd do would be to tell you how to use your own aesthetic 
faculties. (You see? We're unemployed. If not yet, "soon again 'twill be." We 
have nothing to do. So what shall we do? Sit in an audience? Write criticism? 
Be creative?) We used to have the artist up on a pedestal. Now he's no more 
extraordinary than we are. III. Notice audiences at high altitudes and 
audiences in northern countries tend to be attentive during performances 
while audiences at sea level or in warm countries voice their feelings 
whenever they have them. Are we, so to speak, going south in the way we 
experience art? Audience participation? (Having nothing to do, we do it 
nonetheless; our biggest problem is fmding scraps of time in which to get it 
done. Discovery. Awareness.) "Leave the beaten track. You'll see some-thing 
never seen before." After thefirst performance ofmy piecefor twelve radios, 
Virgil Thomson said, "You can't do that sort of thing and expect people to 
payfor it." Separation. IV. When our time was given to physical labor, we 
needed a stifl upper lip and backbone. Now that we're changing our minds, 
intent on things invisible, inaudible, we have other spineless virtues: flexibility, 
fluency. Dreams, daily events, everything gets to and through us. (Art, if you 
want a defmition of it, is criminal action. It conforms to no rules. Not even its 
own. Anyone who ex-periences a work of art is as guilty as the artist. It is not a 
question of sharing the guilt. Each one of us gets all of it.) They asked me 
about theatres in New York. I said we could use them. They should be small for 
the audiences, the performing areas large and spacious, equipped for 
television broadcast for those who prefer staying at home. There should be a 
cafe in connection having food and drink, no music, facilities for playing 
chess. V. What happened at Rochester? We'd no sooner begun playing than 
the audience began. Began what? Costumes. Food. Rolls of toilet paper 
projected in streamers from the balcony through the air. Programs, too, 
folded, then flown. Music, perambulations, conversations. Began festivities. An 
audience can sit quietly or make noises. People can whisper, talk and even 
shout. An audience can sit still or it can get up and move around. People are 
people, not plahls. "Do you love the audience?" Certainly we do. We show it 
by getting out of their way. (Art and money are in this world together, and they 



need each other to keep on going. Perhaps they're both on their way out. 
Money'll be-rnmp a rredit rard without a monthlv bill. What'll art become? A 
familv reunion? 
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If so, let's have it with people in the round, each individual free to lend his at- 
tention wherever he will. Meeting house.) VI. After an Oriental decade, a Tibetan 
Bikku returned to Toronto to teach. He told me that were he to speak the truth 
his audience would drop to six. Instead he gives lectures transmitting not the 
spirit but the understandable letter. Two hundred people listen on each 
occasion, all of them deeply moved. (Art's a way we have for throwing out 
ideas — ones we've picked up in or out of our heads. What's marvelous is 
that as we throw them out — these ideas — they generate others, ones that 
weren't even in our heads to begin with.) Charles Ives had this idea: the 
audience is any one of usjust a human being. He sits in a rocking chair on a 
verandah. Looking out toward the mountains, he sees the setting sun and 
hears his own symphony: it's nothing bui the sounds hauDenine in the air 
around him. 
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'Behaviourist Art and the Cybernetic Vision" 
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RoyAscott. Photo byjosephine 
Coy. Courtesy o/Roy Ascott. 

'Ifthe cybernetic spirit constitutes the predominant 
attitude of the modern era, the computer is the 
supreme tool that jts technology has produced. 
Used in conjunction with synthetic materials it can 
be expected to open up paths of radical change 
and invention in art.... The interaction of man and 
computer in some creative endeavor, involving the 
heightening ofimaginative thought, js to be 
expected. " 
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« Since the 1960s, the British educator, artist, and theoretidan Roy Ascott has 
been one of Europe's most active and outspoken practitioners of interactive 
computer art. Ten years before the personal computer carne into existence, 
Ascott saw that interactivity jn computer-based forms of expression would be an 
emerging jssue in the arts. Intrigued by the possibilities, he built a theoretical 
framework for approaching interactive artworks, which brought together certain 
characteristics of the avant-garde (Dada, surrealism, Fluxus, Happenings, and 
pop art, in particular) with the science of cybernetics championed by Norbert 
Wiener. 

Ascott's thesis on cybernetic vision jn the arts begins with the premise thai 
interactive art must free itself from the modernist ideal of the "perfect object." 
Like John Cage, he proposes that the artwork be responsive to the viewer, rathei- 
than fixed and static. But Ascott expands on Cage's premise jn the realm of 
computer-based art, suggesting that the "spirit of cybernetics" offers the mos1 
effective means for achieving a two-way exchange between the artwork and its 
audience. Ascott challenges artists to acknowledge information technology as the 
most significant tool of the age, and insists that it is the artist's obligation to use 
this technology. Yet, unlike Nam June Paik's vision, Ascott's js not ironic; rather, it 
is Utopian in its embrace of a new medium, excited by the potential of a thriv-ing, 
dynamic exchange between technology and art to empower the spectator and 
deepen his or her experience. » 



the behavioural tendency in modern art 

... By "Modern Art" we mean that cultural continuum of ideas, forms and 
human activity which differs radically from any previous era and is both 
expressive and formative of the altitudes and conditions of our time. To 
describe it as a continuum may seem contradictory to its accepted identity. It is 
seen popularly as an an-archic, highly diversified and chaotic situation which 
loses as much in coherence and continuity as it gains in novelty and 
imagination. 

Now, undoubtedly it is anarchic, but in the good sense that interaction 
between artists is free and not constrained by aesthetic canons or political 



directives. The diversity of images, structures and ideas which it engenders 
is far greater than at any other period in history. And it may well seem chaotic; 
a common cultural consciousness is not readilv aocarent todav. But it is our 
nurnose to demon- 
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strate that Modern Art is fundamentally of a piece, that there is unity in its di- 
versity, and that the quality which unifies it is in distinct contrast to the essential 
nature of the art which went before it. We shall describe this quality as 
"behav-ioural" and we shall show how it evidences our present transition 
from the oid deterministic culture to a future shaped by a Cybernetic Vision. 

The analysis of this behavioural tendency will be largely confined to one 
broad area, that of the visual/plastic arts, since there it seems to be most 
marked, but in a more general sense we shall discuss the arts as a whole, 
illustrating their convergence and interaction in this context. We shall 
demonstrate how this unity of approach may be potentially part of a larger 
unity, an integral culture, em-bracing modern science and technology. And we 
shall warn how this unity, and the incipient cybernetic vision in art, may be 
inhibited by artistic altitudes which, out of ignorance and fear, are opposed to 
radical creative change, and view a cy-bernated society with indifference or 
hostility. 

the general characteristics of modern art 

The dominan! feature of art of the past was the wish to transmit a clearly 
defined message to the spectator as a more or less passive receptor, from the 
artist as a unique and highly individualised source. This deterministic 
aesthetic was cen-tred upon the structuring, or "composition," offacts, of 
concepts of the essence of things, encapsulated in a factually correct visual 
field. Modern Art, by contrast, is concerned to initiate events and with the 
forming of concepts ofexistence. The vision of art has shifted from the field of 
objects to the field of behaviour and its function has become less descriptive 
and more purposive. 

Although in Painting and Sculpture the channel of communication remains 
largely visual, other modalities are increasingly employed — tactile, postural, 
aural; so that a more inclusive term than "visual" art must be found, and the one 
we propose is "behavioural." This behavioural tendency dominates art now in 
all its as-pects. The artist, the artifact and the spectator are all involved in a 
more behavioural context. We find an insistence on polemic, formal ambiguity 



and in-stability, uncertainty and room for change in the images and forms of 
Modern Art. And these factors predominate not for esoteric or obscurantist 
reasons but to draw the spectator into active participation in the act of 
creation; to extend him, via the artifact, the opportunity to become involved in 
creative behaviour on all 
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leveis of experience — physical, emotional and conceptual. A feedback loop is 
es-tablished so that the evolution of the artwork/experience is governed by 
the indmate involvement of the spectator. As the process is open-ended the 
spectator now engages in decision-making play. 



Creative Participation 

We may say that the boundaries between making art, the artifact itself, and the 
experience of the work are no longer clearly defined. Or, more precisely, that 
the tendency for this to be so is evident. There are still in this transitional 
period many artists who contrive to force the new sensibility into oid 
mouldsjust as in technology there are many industrialists who attempt to 
squeeze cybernation into a nineteenth-century structure of operations. 

The participational, inclusive form of art has as its basic principle 
"feedback," and it is this loop which makes of the triad artist/artwork/observer 
an integral whole. For art to switch its role from the private, exclusive arena 
of a rarefied elite to the public, open field of general consciousness, the artist 
has had to create more flexible structures and images ofFering a greater variety 
of readings than were needed in art formerly. This situation, in which the 
artwork exists in a perpetual state of transition where the effort to establish a 
final resolution must come from the observer, may be seen in the context of 
games. We can say that in the past the artist played to win, and so set the 
conditions that he always domi-nated the play. The spectator was positioned 
to lose, in the sense that his moves were predetermined and he could form no 
strategy of his own. Nowadays we are moving towards a situation in which the 
game is never won but remains perpet-ually in a state of play. While the general 
context of the art-experience is set by die artist, its evolution in any specific 



sense is unpredictable and dependent on the total involvement of the 
spectator. 

Where once the function of art was to create an equilibrium, establish a 
har-mony on the public level of relatively passive reception, we now find art 
as a more strident agent of change, effecting ajolt to the whole human 
organism, a cat-alyst which sets up patterns of behaviour, of thought and 
emotion, which are unpredictable in any fine sense. We observe in the 
painting of Poussin, for ex-ample, the wish to fix a set of relationships in the 
spectator's consciousness, te reinforce these absolutes by the stability of the 
formal composition; he commu-nicates but by a one-way channel. The 
modern artist, on the other hand, is pri-marily motivated to initiate a dialogue, 
to set feelings and ideas in motion, te enrich the artistic experience with 
feedback from the spectator's response. 
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This cybernetic process of retraction generales a constant stream of new 
and nnfamiliar relationships, associative links and concepts. Each artwork 
becomes / sort of behavioural Tarot pack, presenting coordinates which can 
be endlessly reshuffled by the spectator, always to produce meaning. This is 
achieved princi-pally in one of two ways: either the artifact has a definitive form 
but contains only a small amount of low-definition information; or its physical 
structure is such that its individual constituent parts can change their 
relationships, either by the direct manipulation of the spectator, or by his 
shiftiiig viewpoint, or by the agency of electrical or other natural power. The 
active involvement of the spectator can be thought of as removing uncertainty 
about a set of possibilities. Deep involvement and interplay produces 
information. The "set" of the artwork has variety only in so far as the observer 
participates. The variety of the set is a measure oi the uncertainty involved. 
An important characteristic of Modern Art, then, is that it offers a high degree 
of uncertainty and permits a great intensity of partic-ipation. 

As to the artist's role, it can be said to function on two leveis simultaneously, 
the private and the social. In the first case, the primacy of a total behavioural 
involvement in the activity or process of making art is apparent. The artist is 
not goal-directed in the sense of working towards a predetermined art object. 
The artifact is essentially the result of his creative behaviour, rather than the 
reason for it. The growth of a painting or sculpture or environment is of more 
importance than the achievement of its final form. Indeed, unlike Classical Art, 
there is no point at which it can be said to have reached a final form. From the 
social point of view the artist's behaviour is a Ritual in which he acts out the 
role of the Free Man controlling his world by taking endless risks as he 
plunges into the un-known territories of Form and Idea. It is a paradigm of a 



condition to which the human being constantly aspires, where freedom and 
responsibility combine to reduce our anxiety of the unknown and unpredictable 
while enlarging our experi-ence of the unfamiliar and irresistible. 

At this early stage of a radically new culture the artist is doing little more than 
exploring his new relationship to the spectator. He is searching for new ways 
oi handling ideas, for more flexible and adaptive structures to contain them; 
he is attempting to generate new carrier waves for the modulations of 
contemporaiy experience; and he is searching the resources of technology to 
expand his reper-toire of skills. His concern is to affirm that dialogue is possible 
— that is the conteni and the message of art now; and that is why, seen from the 
deterministic poinl of view, art may seem devoid of conteni and the artist to have 
nothing to say. The mndern means of nommuniration. of feedback and viable 
interolav — these are th( 
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content of art. The artist's message is that the extension of creative behaviour 
into sveryday experience is possible. 

The message is timely and apposite at a period in which we can anticipate 
the reduction of labour to a minimum and expect the creative use of leisure 
time to be the main preoccupation of our lives. And even if the artist were to 
have fully explored the new channels of communication and thoroughly 
exploited the media and techniques of modern technology, it is unlikely that his 
attitude would change. He would continue to avoid the limitations of an 
aesthetic geared to the transmission of fmite messages or the formulation of 
fixed altitudes and absolute values. He will continue, instead, to provide a 
matrix for ideas and feelings from which the participants in his work may 
construct for themselves new experi-ences and unfamiliar patterns of 
behaviour.... 



the cybernetic vision in art 

By this term we do not mean "the Art of Cybernetics" nor do we refer to an art 
concerned to illustrate Cybernetics, nor yet an art embodying cybernetic 
machines or Robot Art — although any one of these things might be involved 
at some point, and again, they might not. 

We are referring to the spirit of Cybernetics which may inform art and in 
turn be enriched by it. We contrast the Cybernetic Vision in Art to the Deter- 
ministic Vision of the past which has already been outlined. We say of 
Cybernetics that, befo re it is a method or an applied science, it is a field of 



knowledge which shapes our philosophy, influences our behaviour and extends 
our thought. 

We are moving towards a fully cybernated society 1 where processes of re- 
traction, instant communication, autonomic flexibility will inform every aspect oi 
our environment. In that forming society, of which we are a part, the cybernetic 
spirit finds its expression in the Human Science 2 and in Environmental 
Technology; the two poles between which we act out our existence. It is the 
spirit of our understanding of Ufe at its simplest and most complex leveis, 
and a large measure of our ability to control it. 

The economic and social effects of automation in the cybernated society wil] 
be profound. 3 The effects of our transition of that future state are already felt, par- 
ticularly in the United States. Matters of leisure, class formation, political and 
economic power have already called for revision and new thinking. 
Cybernetics already dominates our more advanced concepts of transport, 
shelters, storage and 

nfhpr rlav tn rlav matters nf rnntrnl and rommumration and has ranser] the 
rad- 
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ical transformation of many industrial and commercial procedures. The effect 
of the computer on human thought is currently the subject of vigorous 
discussion in academic circles; the man/computer relationship is seen to be as 
much a ques-tion of identity as of methodology. 

Fundamentally Cybernetics concerns the idea of the perfectibility of 
systems: it is concerned in practice with the procurement of effective action by 
means oi self-organising systems. It recognises the idea of the perfectibility of 
Man, of the possibility of further evolution in the biological and social sphere. In 
this it shares its optimism with Molecular Biology. Bio-cybernetics, the 
simulation of living processes, genetic manipulation, the behavioural 
sciences, automatic environ-ments, together constitute an understanding of 
the human being which calis for and will in time produce new human values 
and a new morality. 

How does the artist stand in relation to these radical changes? On the 
level of opinions or concepts he is and will be free to accept or reject them. But 
on the level of deep human experience they will "alter sense ratios or patterns 
of per-ception steadily and without resistance." 4 The artist is faced with two 
possibili-ties; either to be carried along in the stream of events, mindlessly, half 
aware and perhaps bitter and hostile as a result; or he can come to terms 
with his world, shape it and develop it by understanding its underlying 
cybernetic characteris-tics. Awareness of these underlying forces will sharpen 
his perception; the utili-sation of new techniques will enlarge his powers of 
thought and creative action; he will be empowered to construct a vision in art 



which will enhance the cyber-nated society as much as it will be enriched by it. 
Understanding and awareness, in short, are the conditions for optimism in art. 

There is reason to suppose that a unity of art, science and human values 
is possible; there is no doubt that it is desirable. 5 More specifically we 
propose that an essentially cybernetic vision could unify and feed such 
culture. The grounds for supposing that Art has anticipated this integral 
situation and is pre-pared for it can be found in the emphatically behavioural 
tendency which it dis-plays. Cybernetics is consistent with Behaviourist Art; it 
can assist in its evolution just as, in turn, a behavioural synthesis can embody 
a Cybernetic Vision. 



cybernetics and behaviourist art 

[t is necessary to differentiate between "I'esprit cybernetique" 6 as we have 
tried to describe it above, and Cybernetics as a descriptive method. Now, art 
like any process or system can be examined from the cybernetic point of view; 
it can als 
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derive technical and theoretical support from this science — as in the past it 
has done from Optics or Geometry. This is not unimportant since the artist's 
range can be extended considerably, as we briefly indicate below. But it is 
important to remember that the Cybernetic Vision in Art, which will unify art 
with a cyber-nated society, is a matter of "stance," a fundamental attitude to 
events and human relationships, before it is in any sense a technical or 
procedural matter. 

Behaviourist Art constitutes a retroactive process of human involvement, in 
which the artifact functions as both matrix and catalyst. As matrix, it is the sub- 
stance between two sets of behaviours; it neither exists for itself nor by itself. 
As a catalyst, it triggers changes in the spectator's total behaviour. Its structure 
must be adaptive implicitly or physically, to accommodate the spectator's 
responses, in order that the creative evolution of form and idea may take 
place. The basic principle is feedback. The system Artifact/Observer furnishes 
its own controlling energy; a function of an output variable (observer response) 
is to act as an input variable, which introduces more variety into the system 
and leads to more vari-ety in the output (observer's experience). This rich 
interplay derives from what is a self-organising system in which there are two 



controlling factors; one, the spectator is a self-organising sub-system; the 
other, the artwork is not usually at present homeostatic. 

There is no a priori reason why the artifact should not be a self-organising 
system; an organism, as it were, which derives its initial programme or code 
from the artist's creative activity, and then evolves its specific artistic identity and 
function in response to the environments which it encounters. The artist's 
creative activity is also dependen! on feedback; the changes which he 
effects in his immediate environment (or "arena") by means of tools and 
media set up config-urations which feed back to affect his subsequent 
decisions and actions. Thus Modern Art, with its fundamental behavioural 
quality, is the art of the organisa-tion of effects. And when all the control 
factors, including the artwork itself, are eiFectively homeostatic, art will be 
concerned with the automatic control of effects. Cybernetics, of course, is the 
science of the organisation of effects, and of the automatic control of efFects. 7 

Equally, there is no a priori reason why the artwork should become a self- 
organising system; the basic feedback process of behaviourist art operales 
within the conventions of painting and sculpture, provided that they display low 
defin-ition, multiple associations and indeterminate conteni, within paramelers 
which are, al leasl implicitly, flexible. And, as we have suggested already, ibis is 
nowa-days ihe case — even lo ihe extent of providing a more or less emply 
receplack (ihe canvas) into which the speclalor can projecl his own 
imaginative world, e.e:., Yves Klein, Ad Reinhard. 
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the computer and growth systems 

However, historically it has been a characteristic of the artist to reach out to 
the tools and malcriais which the technology ofhis time producesjust as his 
percep-tion and patterns of thought have tended to identify with scientific and 
philo-sophical altitudes of the period. If ihe cybernelics spiril constitutes the 
predominan! attitude of ihe modern era, the computer is ihe supreme tool ihat 
its lechnology has produced. Used in conjunclion wilh synlhelic malcriais it can 
be expected to open up paths of radical change and invention in arl. For it is not 
sim-ply a physical tool in the sense thai an aluminium casling plant or CO2 welding 
geai are tools, i.e., extensions of physical power. It is a tool for the mind, an 
instrumenl for the magnification of thought, potentially an intelligence amplifier. 8 
The inter-action of man and computer in some creative endeavour, involving 
the heighten-ing of imaginative thought, is to be expected. Moreover the 
interaction of Artifacl and computer, in the context of the behavioural structure, 
is equally foreseeable. 

Experiments are already taking place. We have cited Schoffer's use of a 
computer in some of his structures. In music Iannis Xenakis has made 
extensive use of an IBM 7090 — a process in which he "specifies the duration 



and density oi sound events, leaving the parameters of pitch, velocity and 
dynamics to the computer." The "Light-Harp" project of Haukeland and 
Nordheim, an environ-mental sculpture emitting sound in relation to the 
quality of local light, with sound sources changing position within the 
structure, calis for a highly sophis-ticated control and Communications system 
within it. 

The computer may be linked to an artwork and the artwork may in some 
sense be a computer. The necessary conditions of behaviourist art are that 
the spectator is involved and that the artwork in some way behaves. Now, it 
seems likely that in the artist's attempt to create structures which are 
probabilistic, the artifact may result from biological modelling. In short, it may 
be developed with the properties of growth. Cybernetics already furnishes 
models which could as-sist in this development, e.g., Beer's Fungoid Systems 
and research into chemi-cal and chemical-colloidal computers. 9 The 
potential for the future is 
enormous. 10 

The cybernetic vision not only shapes modern science and technology, inte- 
grating and bridging disparate fields of knowledge and improving artificial 
control and communication systems by the understanding of complex 
natural processes, but it can be expected to find expression and enlargement in 
Art. It can assist in the evolution of art. servine to increase its varietv and 
vieour.. 



nyron mjeger 



1 i, » "Responsiva Environments" (1977) 




Krueger. Courtesy of 
Myron Krueger. 



"The responsive environment has 
been presented as the basis for a 
new aesthetic medium based on real- 
time interaction between men and 
machines. In the long range it augurs 
a new realm of human experience, 
artificial realities which seek not to 
stimulate the physical world but to 
define arbitrary, abstract and 
otherwise Impossible relationships 
between action and result. " 



105 nesponsive hnvironmenfs 



« Myron Krueger was among the first artists to explore the computer as a central 
component jn Interactive art. Originally trained as a computer scientist, Krueger, 
under the influence of John Cage's experiments jn indeterminacy and audience 
participaron, pioneered human-computer interaction jn the context of physical 
environments. Beginning jn 1969, he collaborated with artist and engi-neer 
colleaguesatthe University of Wisconsin to create artworks that responded to the 
movement and gesture of the viewer through an elaborate system of sens-ing 
floors, graphic tables, and video cameras. In this essay, which reflects the 



influence of cybernetics on the arts and sciences, Krueger discusses human- 
computer interaction as the basis for an emerging art form. 

At the heart of Krueger's contribution to interactive computer art was the 
notion of the artist as a "composer" of intel ligent, real-time computer-mediated 
spaces, or "responsive environments," as he called them. Krueger "composed" 
environments jn which the computer responded to the gestures of the audience 
by interpreting, and even anticipating, their actions. In such works as /Wefap/a) 
and Videoplace, both from the 1970s, images of audience members would be pro- 
jected on an eight-by-ten-foot screen; superimposed on the screen would be 
animated figures and special effects designed by the artist. Audience members 
:ould "touch" each other's video-generated silhouettes, as well as manipulate 
the odd, playful assortment of graphical objects and animated organisms that 
appeared on the screen, imbued with the presence of artificial life. 

In Krueger's artworks the images of spectators video-broadcast from separate 
locations would be projected into a single space.This collage effect antici-pated 
subsequent developments in telepresence, which can be described as the state of 
being present simultaneously in more than one location. In these early 
sxplorations of telematic art, audience members experienced a sensation of 
virtual proximity, as if they were standing next to each other jn the same space, 
though they were separated by a significant distance. This aspect of Krueger's 
work makes it a precursor to today's global Communications technology, in- 
cluding video-conferencing, networked virtual worlds, and live on-line chat. » 



introduction 

Man-machine interaction is usually limited to a seated man poking at a 
machine with his fingers or perhaps waving a wand over a data tablet. Seven 
years ago, 1 was dissatisfied with such a restricted dialoeue and embarked 
on research ex- 
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ploring more interesting ways for men and machines to relate. The result was 
the concept of a responsive environment in which a computer perceives the 
actions of those who enter and responds intelligently through complex visual 
and audi-tory displays. 

Over a period of time the computer's displays establish a context within 
which the interaction occurs. It is within this context that the participant 
chooses his next action and anticipates the environment's response. If the 
response is unexpected, the environment has changed the context and the 



participant must reexamine his expectations. The experience is controlled by a 
composition which anticipates the participant's actions and flirts with his 
expectations. 

This paper describes the evolution of these concepts from their primitive 
beginnings to my current project, Videoplace, which provides a general tool foi 
devising many interactions. Based on these examples an interactive art form is 
de-fmed and its promise identified. While the environments described were 
pre-sented with aesthetic intent, their implications go beyond art. In the final 
section applications in education, psychology and psychotherapy are 
suggested. 



glowflow 

In 1969,1 became involved in the development o'iGlowflow, a computer art pro 
ject conceived by Dan Sandin, Jerry Erdman and Richard Venezsky at the Uni 
versity of Wisconsin. It was designed in an atmosphere of encounter between 
ar and technology. The viewer entered a darkened room in which glowing 
lines o light defined an illusory space. The display was accomplished by 
pumping phos phorescent particles through transparent tubes attached to 
the gallery walls These tubes passed through opaque columns concealing 
lights which excited th phosphors. A pressure sensitive pad in front of each of 
the six columns enablei the computer to respond to footsteps by lighting 
different tubes or changing th sounds generated by a Moog synthesizer or the 
origin of these sounds. Howevei the artists' attitude toward the capacity for 
response was ambivalent. They fe! that it was important that the environment 
respond, but not that the audience b aware of it. Delays were introduced 
between the detection of a participant and th computer's response so that the 
contemplative mood of the environment woul not be destroyed by frantic 
attempts to elicit more responses. 

While Glowflow was quite successful visually, it succeeded more as a kineti 
sculpture than as a responsive environment. However, the Glowflow experienc 
led me to a number of decisions: 
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3. Interactive art is potentially a richly composable medium quite distinct 
from the concerns of sculpture, graphic art or music. 



4. In order to respond intelligendy the computer should perceive as much as 
possible about the participant's behavior. 

5. In order to focus on the relationships between the environment and the 
participants, rather than among participants, only a small number ofpeo- 
ple should be involved at a time. 

6. The participants should be aware of how the environment is responding to 
them. 

7. The choice of sound and visual response systems should be dictated 
bytheir ability to convey a wide variety of conceptual relationships. 

8. The visual responses should not be judged as art nor the sounds as 
music.The only aesthetic concern is the quality of the interaction. 



metaplay 

Fbllowing the Glowflow experience, I conceived and directed Metaplay, which 
was exhibited in the Memorial Union Gallery of the University of Wisconsin for 
/ month in 1970. It was supported by the National Science Foundation, the 
Computer Science Department, the Graduate School and the loan of a PDP- 
12 by Digital Equipment Corporation. 

Metaplay\ focus reflected my reactions to Glowflow. Interaction between 
the participants and the environment was emphasized; the computer was 
used to facilitate a unique real-time relationship between the artist and the 
participan!. An 8' X 10' rear-projection video screen dominated the gallery. 
The live video image of the viewer and a computer graphic image drawn by an 
artist, who was in another building, were superimposed on this screen. Both 
the viewer and the artist could respond to the resulting image. 

Hardware 

The image Communications started with an analogue data tablet which 
enabled the artist to draw or write on the computer screen. The person doing 
the draw-ing did not have to be an artist, but the term is used for convenience. 
One video camera, in the Computer Center, was aimed at the display screen 
of the Adage Graphic Display Computer. A second camera, a mile away in the 
gallery, picked ud the live imaee of oeople in the room. A television cable 
transmitted the video 
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computer image from the Computer Center to the gallery and the two signais 
were mixed so that the computer image overlayed the live image. The 
compos-ite image was projected on the 8' X 10' screen in the gallery and was 



simultane-ously transmitted back to the Computer Center where it was 
displayed on a video monitor providing feedback for the artist. 

The artist could draw on the Adage screen using a data tablet. By using 
func-tion switches, potentiometers and the teletype keyboard the pictures 
could be rapidly modified or the mode of drawing itself altered. In addition to 
the eflects of simple drawings, the image could be moved around the screen, 
image size could be controlled and the picture could be repeated up to ten 
times on the screen displaced by variable X, Y and size increments. A tail of a 
fixed number of line segments could be drawn allowing the removal of a 
segment at one end while another was added at the opposite end. An image 
could be rotated in 3-space under control of the pen. Although this was not 
true rotation, the visual ef-fect was similar. A simple set of transformations 
under potentiometer and tablet control yielded apparent animation of people's 
outlines. Finally, previously de-fmed images could be recalled or exploded. 
While it might seem that the drawing could be done without a computer, the 
ability to rapidly erase, recall and transform images required considerable 
processing and created a far more pow-eriiil means of expression than pencil 
and paper could provide. 



Interaction 

These facilities provided a rich repertoire for an unusual dialogue. The artist 
could draw pictures on the participants' images or communicate directly by writ- 
ing words on the screen. He could induce people to play a game like Tic-Tac- 
Toe or play with the act of drawing, starting to draw one kind of picture only to 
have it transformed into another by interpolation. 



Live Grqffiti 

One interaction derived from the artist's ability to draw on the image of the au- 
dience. He could add grafEti-like features or animate a drawn outline of a 
persori so that it appeared to dance to the music in the gallery. The artist tried 
various approaches to involve people in the interaction. Failing to engage one 
person, he would seek someone more responsive. 

It was important to involve the participants in the act of drawing. However 
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the electronic wand designed for this purpose did not work reliably. Whal 
evolved was a serendipitous solution. One day as I was trying to draw on a 



stu-dent's hand, he became confused and moved it. When I erased my 
scribblings and started over, he moved his hand again. He did this repeatedly 
until it became a game. Finally, it degenerated to the point where I was simply 
tracking the image of his hand with the computer line. In effect, by moving his 
hand he could draw on the screen before him. 

The relationship established with this participan! was developed as one of 
the major themes ofMetaplay. It was repeated and varied until it became an 
aesthetic medium in itself. With each person we involved in this way, we tried to 
preserve the pleasure of the original discovery. After playing some graffiti games 
with each group that entered, we would focus on a single individual and draw 
around the image of his hand. After an initial reaction of blank bewilderment, 
the self-conscious person would make a nervous gesture. The computer line 
traced the gesture. A second gesture, followed by the line, was the key to 
discovery. One could draw on the video screen with his finger! Others in the 
group, observing this phenomenon, would want to try it too. The line could be 
passed from one person's finger to another's. Literally hundreds of 
interactive vignettes developed within this simple communication channel. 

Drawing by this method was a rough process. Pictures of any but the 
simplest shapes were unattainable. This was mainly because of the difficulty of 
tracking a person's finger. Happily, neither the artist nor the audience was 
concerned about the quality of the drawings. What was exciting was 
interacting in this novel way through a man-computer-video link spanning a 
mile. 



psychic space 

The next step in the evolution of the responsive environment was Psychic 
Space, which I designed and exhibited in the Memorial Union Gallery during 
May and fuhe of 1971. It was implemented with the help of my students, the 
Computer Science Department and a National Science Foundation grant in 
Complex Information Processing. 

Psychic Space was both an instrument for musical expression and a richly 
composed, interactive, visual experience. Participants could become involved 
in i softshoe duet with the environment, or they could attempt to match wits 
with the computer by walking an unpredictable maze projected on an 8' X 10' 
video 
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Hardware 



A PDP-11 had direct control of all sensing and sound in the gallery. In addition, 
it communicated with the Adage AGT-10 Graphic Display Computer at the 
Computer Center. The Adage image was transmitted over video cable to the 
gallery where it was rear-projected on the 8' X 10' screen. The participant's 
po-sition on the floor was the basis for each of the interactions. The sensing 
was done by a 16' X 24' grid of pressure switches, constructed in 2' X 4' 
modules, each containing 8 switches. Since they were electronically 
independent, the system was able to discriminate among individuals if 
several were present. This inde-pendence made it easy for the programming 
to ignore a faulty switch until its module was replaced or repaired. Since 
there were 16 bits in the input words of the PDP-11, it was natural to read the 
16 switches in each row across the room in parallel. Digital circuitry was then 
used to sean the 24 rows under computer control. 

Input and Interaction 

Since the goal was to encourage the participants to express themselves 
through the environment, the program automatically responded to the 
footsteps of peo-ple entering the room with electronic sound. We 
experimented with a number of difieren! schemes for actually generating the 
sounds based on an analysis of peo-ples' footsteps. In sampling the floor 60 
times per second we discovered that a single footstep consisted of as many 
as four discrete events: lifting the heel, lift-ing the toe, putting the heel down 
and putting the ball of the foot down. The first two were dubbed the 
"unfootstep." We could respond to each footstep or un-footstep as it 
occurred, or we could respond to the person's average position. A number of 
response schemes were tried, but the most pleasing was to start each tone 
only when a new switch was stepped on and then to terminate it on the next 
"unfootstep." Thus it was possible to get silence by jumping, or by lifting one 
foot, or by putting both feet on the same switch. 

Typical reaction to the sounds was instant understanding, followed by a 
rapid-fire sequence of stepsjumps and rolls. This phase was followed by a 
slower, more thoughtful exploration of the environment in which more subtle and 
interesting relationships could be developed. In the second phase, the 
participani would discover that the room was organized with high notes at one 
end and lov* notes at the other. After a while, the keyboard was abruptly rotated 
by 90 degrees 

After a lonser period of time an additional feature carne into play. If the com- 
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puter discovered that a person's behavior was characterized by a short series 
oi steps punctuated by relatively long pauses, it would use the pause to 
establish a new kind of relationship. The sequence of steps was responded to 
with a series of notes as before; however, during the pause the computer 
would repeat these notes again. If the person remained still during the pause, 
the computer assumed that the relationship was understood. The next 
sequence of steps was echoed at a noticeably higher pitch. Subsequent 
sequences were repeated several times with variations each time. This 
interaction was experimental and extremely dif-ficult to introduce clearly with 
feedback alone, i.e., without explicit instructions. The desire was for a man- 
machine dialogue resembling the guitar duel in the film Deliverance. 



maze — A composed environment 

The maze program focused on the interaction between one individual and the 
en-vironment. The participant was lured into attempting to navigate a 
projected maze. The intrigue derived from the maze's responses, a carefiilly 
composed sequence of relations designed to constitute a unique and 
coherent experience. 

Hardware 

The maze itself was not programmed on the PDP-11, but on the Adage 
located a mile away in the Computer Centen The PDP-11 transmitted the 
participant's floor coordinates across an audio cable to the Adage. The data 
was transmitted asynchronously as a serial bit stream of varying pulse 
widths. The Adage gen-erated the maze image which was picked up by a TV 
camera and transmitted via a video cable back to the Union where it was 
rear-screen-projected to a size of 
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Interaction 

The first problem was simply to educate the person to the relationships 
between the floor and the screen. Initially, a diamond with a cross in it 
representing the person's position appeared on the screen. Physical 
movement in the room caused the symbol to move correspondingly on the 
screen. As the participant ap-proached the screen, the symbol moved up. As 
he moved away, it moved down, rhe next step was to induce the person to 
move to the starting point of the maze, 

which had not yet appeared on the screen. To this end, another object was placee 
on the screen at the position which would be the starting point of the maze. Thi 
viewer unavoidably wondered what would happen if he walked his symbol to thi 
object. The arrival of his symbol at the starting point caused the object to vanisl 
and the maze to appear. Thus confronted with the maze, no one questioned thi 
inevitability of walking it. 

Software Boundaries 

Since there were no physical constraints in the gallery, the boundaries of th< 
maze had to be enforced by the computen Each attempt to violate a boundary wa 
foiled by one of many responses in the computer's repertoire. The compute 
could move the line, stretch it elastically, or move the whole maze. The line couli 
disappear, seemingly removing the barrier, except that the rest of the maze woul< 
change simultaneously so no advantage was gained. In addition, the symbol rep 
resenting the person could split in half at the violated boundary, with one ha] 
held stationary while the other half, the alter ego, continued to track movement 
However, no progress could be made until the halves of the symbol were re 
united at the violated boundary. 

Even when the participant was moving legally, there were changes in the pro 
gram contingent upon his position. Several times, as the goal was approached. th< 
maze changed to thwart immediate success. Or, the relationship between th< 
floor and the maze was altered so that movements that once resulted in vertica 
motion now resulted in horizontal motion. Alternatively, the symbol represen! ing 
the participant could remain stationary while the maze moved. 

Ultimately, success was not allowed. When reaching the goal seemed immi 
nent, additional boundaries appeared in front of and behind the symbol, boxinj it 
in. At this point, the maze slowly shrank to nothing. While the goal could no be 
reached, the composed frustration made the route interesting. 



Experience 



The maze experience conveyed a unique set of feelings. The video display spaa 
created a sense of detachment enhanced by the displaced feedback: movemen 
on the horizontal plane of the floor translated onto the vertical plane of the screen 
The popular stereotype of dehumanizing technology seemed fulfilled. However 
the maze idea was engaeine; and people became involved willindv. The lack o 
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any other sensation focused attention completely on this interaction. As the 
ex-perience progressed, their perception of the maze changed. From the 
initial im-pression that it was a problem to solve, they moved to the realization 
that the maze was a vehicle for whimsy, playing with the concept of a maze 
and poking fun at their compulsion to walk it. 



videoplace 

For the past two years I have been working on a project called Videoplace, 
under the aegis of the Space Science and Engineering Center of the University 
of Wis-consin. This work is funded by the National Endowment for the Arts 
and the Wisconsin Arts Board. A preliminary version was exhibited at the 
Milwaukee Art Center for six weeks beginning in October 1975. The 
development of Videoplace is still under way and several more years will be 
required before its potential is fully realized both in terms of implementing the 
enabling hardware and explor-ing its compositional possibilities. 

Videoplace is a conceptual environment with no physical existence. It unites 
people in separate locations in a common visual experience, allowing them to 
in-teract in unexpected ways through the video medium. The term 
Videoplace is based on the premise that the act of communication creates a 
place that consists of all the information that the participants share at that 
moment. When people are in the same room, the physical and 
communication places are the same. When the communicants are 
separated by distance, as in a telephone conversation, there is still a sense 
of being together although sight and touch are not possible. By using television 
instead of telephone, Videoplace seeks to augment this sense of place by 
including vision, physical dimension and a new interpretation of touch. 
Videoplace consists of two or more identical environments which can be ad- 
jacent or hundreds of miles apart. In each environment, a single person walks 
into a darkened room where he fmds himself confronted by an 8' X 10' rear- 
view pro-jection screen. On the screen he sees his own life-size image and the 
image of one or more other people. This is surprising in itself, since he is 
alone in the room. The other images are of people in the other environments. 
They see the same Domposite image on their screens. The visual effect is of 
several people in the same room. By moving around their respective rooms, 
thus moving their imites, the participants can interact within the limitations nf 
rhp 
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It is these apparent limitations that I am currently working to overeome. 
When people are physically together, they can talk, move around the same 
space, manipulate the same objects and touch each other. All of these actions 
would ap-pear to be impossible within the Videoplace. However, the opposite 
is true. The video medium has the potential of being more rich and variable in 
some ways than reality itself. 

It would be easy to allow the participants to talk, although I usually preclude 
this, to force people to focus on the less familiar kinds of interaction that the 
video medium pro vides. A sense of dimension can be created with the help of 
computer graphics, which can define a room or another spatial context withiri 
which the participants appear to move around. Graphics can also furnish this 
space with artificial objects and inhabil it with imaginary organisms. The sense 
of touch would seem to be impossible to duplicate. However, since the 
cameras see each person's image in contrast to a neutral background, it is 
easy to digitize the outline and to determine its orientation on the screen. It is 
also easy to tell iJ one person's image touches another's, or if someone 
touches a computer graphic object. Given this information the computer can 
make the sense of touch effec-tive. It can currently respond with sounds when 
two images touch and will ulti-mately allow a person's image to pick up a 
graphic object and move it about the screen. 

While the participants' bodies are bound by physical laws such as gravity 
their images could be moved around the screen, shrunk, rotated, colorized 
anc keyed together in arbitrary ways. Thus, the full power of video processing 
coulc be used to mediate the interaction and the usual laws of cause and effect 
replacec with alternatives composed by the artist. 

The impact of the experience will derive from the fact that each person ha; 
a very proprietary feeling towards his own image. What happens to his imagi 
happens to him. In fact, when one person's image overlaps another's, there 
is ; psychological sensation akin to touch. In Videoplace, this sensation can 
be en hanced in a number of ways. One image can occlude the other. Both 
images cai disappear where they intersect. Both images can disappear except 
where they in tersect. The intersection of two images can be used to form a 
window into an other scene so two participants have to cooperate to see a 
third. 

Videoplace need not involve more than one participant. It is quite possible t 
create a compelling experience for one person by projecting him into this ima A 
inary domain alone. In fact the hardware/software system underlying 
Videopkn is not conceived as a single work but as a general facility for exploring 
all the po: 
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response Is the medium 

The environments described suggest a new art medium based on a 
commitment to real-time interaction between men and machines. The medium 
is comprised of sensing, display and control systems. It accepts inputs from or 
about the participant and then outputs in a way he can recognize as 
corresponding to his be-havior. The relationship between inputs and outputs 
is arbitrary and variable, allowing the artist to intervene between the 
participant's action and the results perceived. Thus, for example, the 
participant's physical movement can cause sounds or his voice can be used 
to navigate a computer-defined visual space. II is the composition of these 
relationships between action and response that is important. The beauty of 
the visual and aural response is secondary. Response is the medium! 

The distinguishing aspect of the medium is, of course, the fact that it re- 
sponds to the viewer in an interesting way. In order to do this, it must know as 
much as possible about what the participant is doing. It cannot respond intelli- 
§endy if it is unable to distinguish various kinds of behavior as they occur. 

The environment might be able to respond to the participant's position, 
roice volume or pitch, position relative to prior position or the time elapsed 
since the last movement. It could also respond to every third movement, the 
rate of movement, posture, height, colors of clothing or time elapsed since the 
per-son entered the room. If there were several people in the room, it might 
respond to the distance separating them, the average of their positions or the 
computer's ibility to resolve them, i.e., respond differently when they are very 
ciose to-a;ether. 

In more complex interactions like the maze, the computer can create a con- 
text within which the interaction occurs. This context is an artificial reality widiin 
ivhich the artist has complete control of the laws of cause and effect. Thus the 
ac-dons perceived by the hardware sensors are tested for significance within 
the cur-rent context. The computer asks if the person has crossed the 
boundary in the maze or has touched the image of a particular object. At a 
higher level the machine can learn about the individual and judge from its 
past experience with similar individuals just which responses would be most 
ehective. 

Currently, these systems are constrained by the total inability of the corn- 
outer to make certain very useful and for the human very simple perceptual 
judg-ments, such as whether a given individual is a man or a woman or is voune: 
or oid. 
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rhe perceptual system will define the limits of meaningful interaction, for the en- 
kdronment cannot respond to what it cannot perceive. To date the sensing 
sys-tems have included pressure pads, ultrasonics and video digitizing. 

As mentioned before, the actual means of output are not as important in 
this medium as they would be if the form were conceived as solely visual or 
auditory. In fact, it may be desirable that the output not qualify as beautiful in 
any sense, for that would distract from the central theme: the relationship 
established be-tween the observer and the environment. Artists are fully 
capable of producing effective displays in a number of media. This fact is well 
known and to duplicate it produces nothing new. What is not known and 
remains to be tested is the va-lidity of a responsive aesthetic. 

It is necessary that the output media be capable of displaying intelligent, or 
at least composed, reactions, so that the participant knows which of his 
actions provoked it and what the relationship of the response is to his action. 
The pur-pose of the displays is to communicate the relationships that the 
environment is trying to establish. They must be capable of great variation and 
fine control. The response can be expressed in light, sound, mechanical 
movement, or through any means that can be perceived. So far computer 
graphics, video generators, light ar-rays and sound synthesizers have been 
used. 



control and composition 

The control system includes hardware and software control of all inputs and 
outputs as well as processing for decisions that are programmed by the artist. 
He must balance his desire for interesting relationships against the 
commitment to respond in real-time. The simplest responses are little more 
than direct feed-back of the participant's behavior, allowing the environment 
to show offits perceptual system. But far more sophisticated results are 
possible. In fact, a given aggregation of hardware sensors, displays and 
processors can be viewed as an in-strument which can be programmed by 
artists with differing sensitivities to create completely diherent experiences. 
The environment can be thought of in the following ways: 

1. An entity which engages the participant in a dialogue. The environment 
ex-presses itself through light and sound while the participant communicates 



with physical motion. Since the experience is an encounter between 
individuals, it mieht legitimately include greetings, introductions and 
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farewells — all in an abstract rather than literal way. The problem is to pro- 
vide an interesting personality for the environment. 

14. A personal 
amplifier. One individual uses the environment to enhance hisability to 
interact with those within it. To the participants the interaction might appear 
similar to that described above. The result would be limitedby the speed of 
the artist's response but improved by his sensitivity to theparticipants' 
moods. The live drawing interaction in Metaplay could beconsidered an 
example of this approach. 

15. An 
environment which has sub-environments with diherent response re- 
lationships. This space could be inhabited by artificial organisms defined 
either visually or with sound. These creatures can interact with the partic 
ipants as they move about the room. 

16. An amplifier of 
physical position in a real or artificially generated space. Movements around 
the environment would result in much larger apparentmovements in the 
visually represented space. A graphic display computercan be used to 
generate a perspective view of a modelled space as it wouldappear if the 
participant were within it. Movements in the room would result in changes in 
the display, so that by moving only five feet within the environment, the 
participant would appear to have moved fifty feet in thedisplay. The rules 
of the modelled space can be totally arbitrary and phys-ically impossible, 
e.g., a space where objects recede when you approachthem. 

17. An instrument 
which the participants play by moving about the space. In Psychic Space 
the floor was used as a keyboard for a simple musical instrument. 

18. A means of 
turning the participant's body into an instrument. His physical posture 
would be determined from a digitized video image and the ori-entation of 
the limbs would be used to control lights and sounds. 

7. A game between the computer and the participant. This variation is reallya 
far more involving extension of the pinball machine, already the most 
commercially successful interactive environment. 

3. An experimental parable where the theme is illustrated by the things that 
happen to the protagonist — the participant. Viewed from this perspective, 
the maze in Psychic Space becomes pregnant with meaning. It was 
impossible to succeed, to solve the maze. This could be a frustrating 
experience if one were trying to reach the goal. If, on the other hand, the 



participant maintained an active curiosity about how the maze would thwart 
him next, the experience was entertainine. Such ooetic corrmositinn nf 
evnpripnrp i« 
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one of the most promising lines of development to be pursued with the en- 
vironments. 



[mplications of the art form 

Por the artist the environment augurs new relationships with his audience and 
his irt. He operales at a metalevel. The participant provides the direct 
performance of the experience. The environmental hardware is the 
instrument. The com-puter acts much as an orchestra conductor controlling 
the broad relationships while the artist provides the score to which both 
performer and conductor are bound. This relationship may be a familiar one 
for the musical composer, al-though even he is accustomed to being able to 
recognize one of his pieces, no matter who is interpreting it. But the artist's 
responsibilities here become even broader than those of a composer who 
typically defines a detailed sequence o: events. He is composing a sequence 
of possibilities, many of which will not be re-alized for any given participant who 
fails to take the particular path along which they lie. 

Since the artist is not dedicated to the idea that his entire piece be expen- 
enced he can deal with contingencies. He can try different approaches, 
dirFeren: ways of trying to elicit participation. He can take into account the 
differences among people. In the past, art has often been a one-shot, hit-or- 
miss proposition. A painting could accept any attention paid it, but could do 
little to maintain in-terest once it had started to wane. In an environment the 
loss of attention can be sensed as a person walks away. The medium can try 
to regain attention and upor. failure, try again. The piece has a second strike 
capability. In fact it can learn te improve its performance, responding not only 
to the moment but also to the entire history of its experience. 

In the environment, the participant is confronted with a completely new kind 
of experience. He is stripped of his informed expectations and forced to deal 
with the moment in its own terms. He is actively involved, discovering that his 
limbs have been given new meaning and that he can express himself in new 
ways. He does not simply admire the work of the artist; he shares in its 
creation. The experience he achieves will be unique to his movements and 
may go beyond the in-tentions of the artist or his understanding of the 
possibilities of the piece. 



Finally, in an exciting and frightening way, the environments dramatize the 
ex-tent to which we are savages in a world of our own creation. The layman 
has ex-tremelv little ahilitv to define the limits of what is oossible with 
curren: 
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technology and so will accept all sorts of cues as representing relationships 
which in fact do not exist. The constant birth of such superstitions indicates 
how much we have already accomplished in mastering our natural environment 
and how dif-Scult the initial discoveries must have been. 



applications 

The responsive environment is not limited to aesthetic expression. It is a 
potent tool with applications in many fields. Videoplace clearly generalizes the 
act oi telecommunication. It creales a form of communication so powerful that 
two people might choose to meet visually, even if it were possible for them to 
mee) physically. While it is not immediately obvious that Videoplace is the 
optimum means of telecommunication, it is reasonably fair to say that it 
provides an infi-nitely richer interaction than Picturephone allows. It broadens 
the range of pos-dbilities beyond current efforts at teleconferencing. Even in 
its fetal stage. Videoplace is far more flexible than the telephone is after one 
hundred years of de-relopment. At a time when the cost of transportation is 
increasing and fiber op-tics promise to reduce the cost of communication, it 
seems appropriate to research the act of communication in an intuitive sense 
as well as in the strictly scientific and problem-solving approaches that 
prevail today.... 



conclusion 

The responsive environment has been presented as the basis for a new 
aesthetic medium based on real-time interaction between men and machines. 
In the long range it augurs a new realm of human experience, artificial realities 
which seek not to simulate the physical world but to define arbitrary, abstract 
and otherwise impossible relationships between action and result. In addition, 
it has been sug-2;ested that the concepts and tools of the responsive 
environments can be fruit-fully applied in a number of fields. 

What perhaps has been obscured is that these concepts are the result of a 
personal need to understand and express the essence of the computer in 
humanis-ic terms. An earlier project to teach people how to use the 



computer was iandoned in favor of exhibits which taught people about the 
computer by let-ng them experience it. Metaplay, Psychic Space and 
Videoplace were designed >communicate an affirmative vision oftechnoloev 
to the \av oublic. This level 



120 nyron m ueyei 

of education is importan!, for our culture cannot continue if a large proportion of 
our population is hostile to the tools that define it. 

We are incredibly attuned to the idea that the sole purpose of our technolog} is to 
solve problems. It also creales concepts and philosophy. We must more fulr; 
explore these aspects of our inventions, because the next generation of technol 
ogy will speak to us, understand us, and perceive our behavior. It will enter ever 
home and office and intercede between us and much of the information and e> 
perience we receive. The design of such intimate technology is an aesthelic issu 
as much as an engineering one. We must recognize this if we are to understan 
and choose what we become as a result of what we have made. 



Alan Kay 

"User InterFace: A Personal View" 



"Therefore, let me argue that the actual dawn of user interface 
design first happened when computer designers finally noticed, 
notjust that end users had functioning minds, but that a better 
understand'mg ofhow those minds worked would completely 
shift the paradigm of interaction. " 



(1989) 




Alan Kay. Courtesy of Alan Kay. 
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During the late 1960s, while a graduate student at the University of Utah Man 
Kay studied with Ivan Sutherland, the pioneering scientist whose worl launched 
the field of Interactive computer graphics. In 1968 he attended Doug las 
Engelbart's historie presentation of the oN Une System, which introduced thi 
mouse to computing, suggesting for the first time the possibility of "navigating' 
information space. In the early 1970s, after forming the Learning Researcl 
Group atthe newly founded Xerox PARC (Palo Alto Research Center), Kay syn 
thesized these influences jnto what is considered the most crucial advancemen of 
human-computer interactivity, the graphical user interface (GUI). Kay devel oped 
the jdea of iconic, graphical representations of computing functions — th> folders, 
menus, and overlapping Windows found on the desktop — based on hi research 
into the intuitive processes of learning and creativity. 

The GUI was an entirely new approach to Interactive computing.The funda 
mentais of the computer interface had received scant attention since the birth c 
the first digital computer, ENIAC, twenty-five years before. At the time, com 
puters were programmed and operated almost exclusively by scientists. Kay re 
alized early on that designing an intuitive, easy-to-use interface required a subtl 
understanding of the dynamics of human perception. IMorbert Wiener's concep of 
the digital computer had been modeled after the brain, with its sophisticate 
network of message passing and synoptic relays. Kay went deeper jn this direc 
tion. He drew on the theories of psychologists Jean Piaget, Seymour Papert, an 
Jerome Bruner, who had studied the intuitive capacities for learning present i 
the child's mind, and the role that images and symbols play jn the building c 
complex concepts. Kay carne to understand, as he put jt, that "doing with image 
makes symbols.'This was the premise behind the GUI, which enabled compul ers 
users to formulate ideas jn real time by manipulating icons on the compute 
screen. 

Kay's approach made computers accessible to nonspecialists. More impoi 
tant, it transformed the computer into a vehicle for popular creative expressiot In 
this essay, Kay describes his realization that the computer was not just 
computational device but a medium jn its own right. Computers, Kay recoc 



nized, might one day replace books.This led him to build a prototype for a pei 
sonal computer, the Dynabook, which would be capable of dynamical 
representing images and concepts. This notebook-sized computer, able to provic 
unprecedented access to information, images, sound, and animation, was a re' 
olutionary departure not only from the stasis of the book but also from the pa 
sivity of television. While the Dynabook was never built, it led to the Xerox Alt the 
first true multimedia machine. » 
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Wnen I was asked to write this chapter, my first reaction was "A book on user 
interface design — does that mean it's now a real subject?" Well, as of 1989, 
it's still yes and no. User interface has certainly been a hot topic for discussion 
since the advent of the Macintosh. Everyone seems to want user interface but 
they are not sure whether they should order it by the yard or by the ton. Many 
are just now discovering that user interface is not a sandwich spread — applying 
the Macintosh style to poorly designed applications and machines is like trying 
to put Bearnaise sauce on a hotdog! 

Of course the practice of user interface design has been around at least 
since humans invented tools. The unknown designer who first put a haft on 
a hand axe was trying not just to increase leverage but also to make it an 
extension of the arm, not just the fist. The evolutionary designer whom 
Richard Dawkins calis the Blind Watchmaker has been at it much longer; all of 
life's star-tling interfitness is the result. A more recent byproduct of the 
industrial revo-lution called ergonomics in Europe and human factors in the 
U.S. has studied how the human body uses senses and limbs to work with 
tools. From the ear-liest use of interactive computing in the fifties — mostly for 
air traffic control and defense — there have been attempts at user interface 
design and application of ergonomic principles. Many familiar components of 
modern user interface design appeared in the fifties and early sixties, 
including pointing devices, win-dows, menus, icons, gesture recognition, 
hypermedia, the first personal com-puter, and more. There was even a 
beautifully designed user interface for an end-user system in JOSS — but its 
significance was appreciated only by its designer and users. 

Therefore, let me argue that the actual dawn of user interface design first 
happened when computer designers finally noticed, not just that end users 
had functioning minds, but that a better understanding of how those minds 
worked would completely shift the paradigm of interaction. 

This enormous change in point of view happened to many computerists in 
the late sixties, especially in the ARPA research community. Everyone had theii 



own catalyst. For me it was the FLEX machine, an early desktop personal 
computer of the late sixties designed by Ed Cheadle and myself. 

Based on much previous work by others, it had a tablet as a pointing 
device, a high-resolution display for text and animated graphics, and multiple 
windows and it directly executed a high-level object-oriented end-user 
simulation lan-guage. And of course it had a "user interface," but one that 
repelled end users in-stead of drawing them closer to the hearth. I recently 
revisited the FLEX machint 

design and was surprised to find how modern its components were — even a 
use of icon-like structures to access previous work. 

But the combination of ingredients didn't gel. It was like trying to bake a pie 
from random ingredients in a kitchen: baloney instead of apples, ground-up 
Cheerios instead of flour, etc. 
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Then, starting in the summer of 1968,1 got hit on the head randomly but 
re-peatedly by some really nifty work. The first was just a little piece of glass at 
the University of Illinois. But the glass had tiny glowing dots that showed text 
char-acters. It was the first flat-screen display. I and several other grad 
students won-dered when the surface could become large and inexpensive 
enough to be a useful display. We also wondered when the FLEX machine 
silicon could become small enough to fit on the back of the display. The 
answer to both seemed to be the late seventies or early eighties. Then we 
could all have an inexpensive powerful notebook computer — I called it a 
"personal computer" then, but I was thinking intimacy. 

I read McLuhan's Understanding Media [1964] and understood that the 
most important thing about any Communications medium is that message re- 
ceipt is really message recovery; anyone who wishes to receive a message 
em-bedded in a medium must first have internalized the medium so it can 
be "subtracted" out to leave the message behind. When he said "the medium 
is the message" he meant that you have to become the medium if you use it. 

That's pretty scary. It means that even though humans are the animais 
that shape tools, it is in the nature of tools and man that learning to use tools 
reshapes us. So the "message" of the printed book is, first, its availability to 
individuals, henee, its potential detachment from extant social processes; 
second, the uni-formity, even coldness, of noniconic type, which detaches 
readers from the vivid-ness of the now and the slavery of commonsense 
thought to propel them into a far more abstract realm in which ideas that don't 
have easy visualizations can be treated. 

McLuhan's claim that the printing press was the dominant force that trans- 
formed the hermeneutic Middle Ages into our scientific society should not be 
taken too lightly — especially because the main point is that the press didn't do 
it just by making books more available, it did it by changing the thought 
patterns of those who learned to read. 



Though much of what McLuhan wrote was obscure and arguable, the sum 
total to me was a shock that reverberates even now. The computer is a 
medium! I had always thought of it as a tool, perhaps a vehicle — a much 
weaker concep-tion. What McLuhan was savine: is that if the personal 
computer is a truly ne\\ 
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medium then the very use of it would actually change the thought patterns of an 
entire civilization. He had certainly been right about the effects of the electronic 
stained-glass window that was television — a remedievalizing tribal influence at 
best. The intensely interactive and involving nature of the personal computer 
seemed an antiparticle that could annihilate the passive boredom invoked by 
television. But it also promised to surpass the book to bring about a new kind of 
renaissance by going beyond static representations to dynamic simulation. What 
kind of a thinker would you become if you grew up with an active simulator con- 
nected, not just to one point of view, but to all the points of view of the ages rep- 
resented so they could be dynamically tried out and compared? I named the 
notebook-sized computer idea the Dynabook to capture McLuhan's metaphor 
in the silicon to come. 

Shortly after reading McLuhan, I visited Wally Feurzeig, Seymour Papert, and 
Cynthia Solomon at one of the earliest LOGO tests within a school. I was amazed 
to see children writing programs (often recursive) that generated poetry, created 
arithmetic environments, and translated English into Pig Latin. And they were 
just starting to work with the new wastepaper-basket-sized turtle that roamed 
over sheets of butcher paper making drawings with its pen. 

I was possessed by the analogy between print literacy and LOGO. While 
designing the FLEX machine I had believed that end users needed to be able to 
program before the computer could become truly theirs — but here was a real 
demonstration, and with children! The ability to "read" a medium means you can 
access malcriais and tools created by others. The ability to "write" in a medium 
means you can generate malcriais and tools for others. You must have both lo be 
lilerale. In prinl writing, the tools you generate are rhelorical; ihey demonslrate 
and convince. In computer writing, the tools you generate are processes; they 
simulate and decide. 

If the computer is only a vehicle, perhaps you can wait unlil high school lo 
give "driver's ed" on il — but if it's a medium, then it must be extended all ihe way 



inlo the world of the child. How to do it? Of course it has to be done on the in- 
limale nolebook-sized Dynabook! Bui how would anyone "read" ihe Dynabook, lei 
alone "wrile" on il? 

LOGO showed ihal a special language designed with ihe end user's charac- 
lerislics in mind could be more successful ihan a random hack. How had Papert 
learned aboul the nature of children's ihoughl? From Jean Piaget, ihe doyen of 
European cognilive psychologist. One of his most imporlanl conlribulions is ihe 
idea ihat children go dirough several distinctive intellectual slages as ihey develop 
frr>m birth to maturilv. Much can be accomplished if the nalure of the stages is 
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beeded and much grief to the child can be caused if the stages are ignored. 
Piaget noticed a kinesthetic stage, a visual stage, and a symbolic stage. An 
example is that children in the visual stage, when shown a squat glass of water 
poured into a tall thin one, will say there is more water in the tall thin one even 
though the pour-ing was done in front of their eyes. 

One of the ways Papert used Piaget's ideas was to realize that young 
children are not well equiped to do "standard" symbolic mathematics until the 
age of 11 or 12, but that even very young children can do other kinds of 
math, even ad-vanced math such as topology and differential geometry, when 
it is presented in a form that is well matched to their current thinking 
processes. The LOGO tur-tle with its local coordinate system (like the child, it 
is always at the center of its universe) became a highly successful "microworld" 
for exploring ideas in differential geometry. 

This approach made a big impression on me and got me to read many 
more psychology books. Most (including Piaget's) were not very useful, but 
then I discoveredjerome Bruner's Towards a Theory of Instruction [1966]. He 
had re-peated and verified many of Piaget's results, and in the process carne 
up with a different and much more powerful way to interpret what Piaget had 
seen. For example, in the water-pouring experiment, after the child asserted 
there was more water in die tall thin glass, Bruner covered it up with a card and 
asked again. This time the child said, "There must be the same because 
where would the water go?" When Bruner took away the card to again reveal 
the tall thin glass, the child immediately changed back to saying there was 
more water. 

When the cardboard was again interposed the child changed yet again. It 
was as though one set of processes was doing the reasoning when the child 



could see the situation, and another set was employed when the child could 
not see. Bruner's interpretacion of experiments like these is one of the most 
importan! foundations for human-related design. Our mentalium seems to be 
made up oi multiple separate mentalities with very different characteristics. 
They reason dif-ferently, have different skills, and often are in conflict. Bruner 
identified a separate mentality with each of Piaget's stages: he called them 
enactive, iconic, symbolic. While not ignoring the existence of other 
mentalities, he eoncentrated on these three to come up with what are still 
some of the strongest ideas for ere-ating learning-rich environments. 
The work of Papert convinced me that whatever user interface design mighl 
be, it was solidly intertwined with learning. Bruner convinced me that learning 
takes place best environmentally and roughly in stage order — it is best to learn 
somethine: kinestheticallv, then iconically, and finally the intuiuve knowledge wil] 
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be in place that will allow the more powerful but less vivid symbolic processes to 
work at their strongest. This led me over the years to the pioneers of environ- 
mental learning: Montessori Method, Suzuki Violin, and Tim Gallwey's Inner 
Game of Tennis, to namejust a few. 

My point here is that as soon as I was ready to look deeply at the human 
el-ement, and especially after being convinced that the heart of the matter lay 
with Bruner's multiple mentality model, I found the knowledge landscape 
positively festooned with already accomplished useful work. It was like the 
man in Moliere's Bourgeois Gentilhomme who discovered that all his life he 
had been speaking prose! I suddenly remembered McLuhan: "I don't know 
who discovered water but it wasn't a fish." Because it is in part the duty of 
consciousness to represent ourselves to ourselves as simply as possible, we 
should sorely distrust our commonsense self view. It is likely that this mirrors- 
within-mirrors problem in which we run into a misleading commonsense notion 
about ourselves at every turn is what forced psychology to be one of the most 
recent sciences — if indeed it yet is. 

Now, if we agree with the evidence that the human cognitive facilities are 
made up of a doing mentality, an image mentality, and a symbolic mentality, 
then any user interface we construct should at least caler to the mechanisms 
that seem to be there. But how? One approach is to realize that no single 
mentality offers a complete answer to the entire range of thinking and problem 
solving. User interface design should integrate them at least as well as Bruner 
did in his spiral curriculum ideas. 

One of the implications of the Piaget-Bruner decomposition is that the 
mentalities originated at very different evolutionary times and there is little 
probability that they can intercommunicate and synergize in more than the 



most rudimen-tary fashion. In fact, the mentalities are more likely to interfere 
with each other as they compete for control. The study by Hadamard on math 
and science creativ-ity [1945] and others on music and the arts indicate 
strongly that creativity in these areas is not at all linked to the symbolic 
mentality (as most theories of teaching suppose), but that the important work 
in creative areas is done in the ini-tial two mentalities — most in the iconic (or 
figurative) and quite a bit in the en-active. The groundbreaking work by Tim 
Gallwey on the "inner game" [1974] showed what could be done if 
interference were removed (mentalities not rele-vant to the learning were 
distracted) and attention was facilitated (the mentalities that could actually do 
the learning were focused more strongly on the environ-ment). 
Finallv. in the sixties a number of studies showed just how modeful was a 
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mentality thathad "seized control" — particularly the analytical-problem-solving 
one (which identifies most strongly with the Bruner symbolic mentality). For 
example, after working on five analytic tasks in a row, if a problem was given that 
was trivial to solve figuratively, the solver could be blocked for hours trying to 
solve it symbolically. This makes quite a bit of sense when you consider that 
the main jobs of the three mentalities are: 

enactive know where you are, manipulateiconic recognize, 
compare, configure, concrete 

symbolic de together long chains of reasoning, abstract 

The visual system's main Job is to be interested in everything in a scene, 
to dart over it as one does with a bulletin board, to change context. The 
symbolic system's main job is to stay with a context and to make indirect 
connections. Imagine what it would be like if it were reversed. If the visual 
system looked at the object it first saw in the morning for five hours straight! Or 
if the symbolic system couldn't hold a thought for more than a few seconds at 
a time! 

It is easy to see that one of the main reasons that the figurative system is so 
cre-ative is that it tends not to get blocked because of the constant flitting and 
dart-ing. The chance of finding an interesting pattern is very high. It is not 
surprising, either, that many people who are "figurative" have extreme dihiculty 
getting any-thing finished — there is always something new and interesting 
that pops up to distract. Conversely, the "symbolic" person is good at getting 
things done, because of the long focus on single contexts, but has a hard time 
being creative, or even being a good problem solver, because of the 
extreme tendency to get blocked. In other words, because none of the 
mentalities is supremely useful to the exclusion of the others, the best 



strategy would be to try to gently force syn-ergy between them in the user 
interface design. 

Out of all this carne the main slogan 1 coined to express this goal: 

Doing with Images makes Symbols 

The slogan also implies — as did Bruner — that one should start with — be 
grounded in — the concrete "Doing with Images," and be carried into the 
more abstract "makes Symbols." 

All the ingredients were already around. We were ready to notice what the 
theoretical frameworks from other fields of Bruner, Gallwey, and others were 
trying to tell us. What is surprising to me is just how long it took to put it all 
together. After Xerox PARC provided the opportunity to turn these ideas into 
reality, it still took our tcrouD about five vears and experiments with hundreds of 
users to come 
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up with the first practical design that was in accord with Bruner's model and re- 
illy worked. 



DOING 



Mouse enactive know where you are, manipulate 



with IMAGES 



icons, windows iconic recognize, compare, configure, concrete 



makes SYMBOLS 

Smalltalk symbolic de together long chains of reasoning, abstract 

Part of the reason perhaps was that the theory was much better at confirm- 
ing that an idea was good than at actually generating the ideas. In fact, in 



certain areas like "iconic programming," it actually held back progress, for 
example, the simple use of icons as signs, because the siren's song of trying 
to do symbolic thinking iconically was just too strong. 

Some of the smaller areas were obvious and found their place in the 
frame-work immediately. Probably the most intuitive was the idea of multiple 
overlap-ping windows. NLS had multiple panes, FLEX had multiple windows, 
and the bit-map display that we thought was too small, but that was made 
from individual pixels, led quickly to the idea that windows could appear to 
overlap. The con-trastive ideas of Bruner suggested that there should always 
be a way to compare. The flitting-about nature of the iconic mentality 
suggested that having as many resources showing on the screen as possible 
would be a good way to encourage creativity and problem solving and prevent 
blockage. An intuitive way to use the windows was to activate the window that 
the mouse was in and bring it to the "top." This interaction was modeless in a 
special sense of the word. The active window 



constituted a mode to be sure — one window might hold a painting kit, 
another might hold text — but one could get to the next window to do 
something in without any special termination. This is what modeless carne to 
mean for medie user could always get to the next thing desired without any 
backing out. The contrast of the nice modeless interactions of windows with the 
clumsy command syntax of most previous systems directly suggested that 
everything should be made modeless. Thus began a campaign to "get rid of 
modes." 

The object-oriented nature of Smalltalk was very suggestive. For example. 
object-oriented means that the object knows what it can do. In the abstract 
sym-holir arena, it means we should first write the object's name (or whatever 
will 
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fetch it) and then follow with a message it can understand that asks it to do 
some-thing. In the concrete user-interface arena, it suggests that we should 
select the object first. It can then furnish us with a menu of what it is willing to 
do. In both cases we have the object first and the desire second. This unirles 
the concrete with the abstract in a highly satisfying way. 

The most difEcult area to get to be modeless was a very uny one, that of 
ele-mentaiy text editing. How to get rid of "insert" and "replace" modes that 
had plagued a decade of editors? Several people arrived at the solution 
simultane-ously. My route carne as the result of several beginning 
programmer adults who were having trouble building a paragraph editor in 
Smalltalk, a problem I thought should be easy. Over a weekend I built a 
sample paragraph editor whose main simplification was that it eliminated the 
distinction between insert, replace, and delete by allowing selections to extend 
between the characters. Thus, there could be a zero-width selection, and thus 
every operation could be a replace. "Insert meant replace the zero-width 
selection. "Delete" meant replace the selection with a zero-width string of 
characters. I got the tiny one-page program running in Smalltalk and carne in 
crowing over the victory. Larry Tesler thought it was great and showed me the 
idea, already working in his new Gypsy editor (which he implemented on the 
basis of a suggestion from Peter Deutsch). So much for creativity and 
invention when ideas are in the air. As Goethe noted, the most im-portant thing 



is to enjoy the thrill of discovery rather than to make vain attempts to claim 
priority!... 

The only stumbling place for this onrushing braver new world is that all oi 
its marvels will be very difficult to communicate with, because, as always, the 
user interface design that could make it all simple lags far, far behind. If 
communica-tion is the watchword, then what do we communicate with and how 
do we do it? We communicate with: 

3. Ourselves 

4. Our tools 

5. Our colleagues and others 

6. Our agents 

Until now, personal computing has concentrated mostly on the first two. Lef 
us now extend every thing we do to be part of a grand collaboration — with one'i 
self, one's tools, other humans, and, increasingly, with agents: computer 
processei that act as guide, as coach, and as amanuensis. The user interface 
design will bt the critical factor in the success of this new way to work and 
play on the com 

puter. One of the implications is that the "network" will not be seen at all, but 
rather "felt" as a shift in capacity and range from that experienced via one's 
own hard disk.... 



Well, there are so many more new issues that must be explored as well. I 
say thank goodness for that. How do we navigate in once-again uncharted 
waters? I have always believed that of all the ways to approach the future, the 
vehicle that gets you to the most interesting places is Romance. The notion of 
tool has always been a romantic idea to humankind — from swords to musical 
instruments to personal computers, it has been easy to say: "The best way to 
predict the future is to invent it!" The romantic dream of "how nice it would be 
if..." often has the power to bring the vision to life. Though the notion of 
management of complex processes has less cachet than that of the hero 
singlehandedly wielding a sword, the real romance of management is nothing 
less than the creation of civilizador] itself. What a strange and interesting 
frontier to investigate. As always, the strongest weapon we have to explore 
this new world is the one between oui ears — orovidine it's loaded! 



JeFFreyShaw 



"ModalitiesoF 
Interactivity and 
Virtuality" (1992) 



Viewer on bicyclefrom The Legible City. Courtesy ofjeffrey Shaw. 

"The traditional activity of art has been the 
representation ofreality-manipulating 
materials to create tangible mirrors ofour 
experience and desire. Now with the 
mechanisms ofthe new digital technologies, 



the artwork can become jtselfa simulation 
ofreality-an im material digital structure 
encompass'mg synthetic spaces, which we 
can literally enter. " 
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« Since the 1960s, Australian-born artist Jeffrey Shaw has been exploring the 
'elationship between interactivity and modes of perception jn his performance 
art, sculpture, videos,and interactive media installations. His interest hasfocused 
on how perception can be affected by interactive environments that free the par- 
ticipant from a single, fixed point of view.This work led him to explore the vari- 
ous ways that we might view, interact with, and manipulate virtual space.This, in 
turn, brought Shaw to ZKM (Center for New Media) jn Karlsruhe, Germany, 
where he has served as director of the Institute of Visual Media since 1991. 

It was in the 1980s that Shaw began to incorporate virtual reality into his 
installations.These early works allowed the viewer/participant to move through 
virtual space by means of an imaginative user-interface device, which Shaw cre- 
ated specifically for each installation. For example, in his 1987 interactive 
videodisc sculpture Inventenla Terre, the participant peers into a panorama through 
a periscope. In his landmark work The Legible City, from 1989, the viewer interacts 
jn real time with projections of an abstract cityscape while riding a stationary 
bicycle; the viewer's perspective would change as he turned left, right, or carne 
to a stop. Shaw's work expanded on the responsive environments of Myron 
Krueger while exploring possibilities for human-computer interaction that moved 
beyond the confines of the computer screen. They placed the participant inside 



the virtual environment and, crucially, allowed her to navigate that space through 
gesture rather than with a mouse. Shaw's work proposed user interface to be a 
central aspect of the artwork's aesthetic. » 

The research and development of various mechanisms and codes of 
spatial representation has been a basic preoccupation throughout the history 
of Western art. The formulation of a set of spatial coord males provides an 
underlying aesthetic and existential paradigm within which a culture 
achieves tentative representation (and thus comprehension) of its desires. 
The recently developed digital imaging technologies offer the artist new 
methods and new paradigms which extend the spatial identity of die artwork. 
And not just in terms of the structure of the image itself, but also in terms of a 
space of interaction between the image and the spectator. Referring to recent 
works I have made, I will describe certain importan! characteristics of the digital 
image, in particular those aspects whirh are ahle to constitute the modalities of 
interactivitv and virtualitv in an art- 
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Painted images hang on walls or rest in storage; bringing them into view is a 
material handling. Digital pictures reside immaterially inside the computer, and 
the computer screen functions like a window through which the viewer 
chooses what he wants to look at. Furthermore, the computer screen functions 
like a cinema camera, because the viewer can pan in any direction over the 
surface of an image and also zoom into the details of a chosen image. These 
characteristics ofFer the possibility to create a virtual space of imagery 
wherein a three-dimensional structure of relationships between two- 
dimensional images can be defined. This can then constitute an interactive 
space which the viewer explores by utilizing some kind of interface device. 
In my interactive work The Narrative Landscape (Aorta, Amsterdam, 1985). 
twenty-eight images were arranged on four leveis, each image having a 
specific spatial location beside, above, and below each other. Using a joystick, 
the viewei could determine an individual path of movement through this 
virtual space o: imagery. This joystick gave the viewer interactive control over 
the movement o the viewing window, which could be panned in any horizontal 
direction over tht surface of an image, and could also be zoomed vertically up 
and down into tht digital detail of an image. This vertical zoom movement also 



allowed the viewei to penetrate through the image surface, and pass from 
one image level to an other. Starting from the upper image level the viewer 
was offered nine difieren pictorial places to enter, each place constituted by a 
narrative group of threi other images. 

The interactive structure of The Narrative Landscape shows that the spatia 
boundaries of the digital image does not have to be defined by the traditiona 
perimeter of a "picture frame." Instead, a virtual image space of any 
dimensioi may be created, which the viewer explores by moving his "viewing 
window.' And as in the cinema, the externalization of the image is always 
referential to th activity of the viewer's eye (the "camera"). 
This location of the viewer at the focal plane of an extended virtual space o 
imagery was also the modality of my installation Going to the Heart ofthe Cenit 
ofthe Carden ofDelights (Vleeshal, Middelburg, 1986). In this installation, imag 
changes occurred as a consequence ofthe viewer walking through the 
museun: from the entrance toward a large video-projected image at the 
opposite ene These image changes were triggered by infrared sensors placed 
along a walkwa that was defined by two rows of blue "airport lights" on the 
floor ofthe museun Furthermore, the sequence of seven image changes was 
defined in relation t the linear progression of seven vaulted compartments that 
characterized this mi seum's Gothic architecture. Thus the museum was 
divided into seven zone 



"Modalities of Interactivity and Virtualiiy" 135 

.vliich constituted the spatial coordinates of seven virtual image planes that 
the idewer passed through when "walking through" this work. 

This particular strategy of locating two-dimensional images in a virtual three- 
dimensional space was also a central aspect of my interactive videodisc 
sculpture Inventer la Terre (La Vulette, Paris, 1987). This work inherits and 
dematerializes the tradition of trompe I'oeil panorama painting — the panorama 
here exists as a true virtual image, positioned around the viewer and 
superimposed into the actual space ofthe museum. A periscope-like sculpture 
contains video and optical mechanisms that enable the viewer to interactively 
roiate his window of view in-side this panorama, and ihen choose videodisc 
sequences ihal are "visits" lo "places" that are represented in that 
panorama. 

The three works so far described demonstrate modalities for the location 
of two-dimensional digital imagery in a virtual three-dimensional space. The 



Nar-rative Landscape had a specific struclure of image leveis sel behind ihe 
plane of the projection screen. Going to the Heart ofthe Center ofthe Carden 
ofDelights implicilly broughl ihese leveis oul in fronl of ihe screen and into the 
room where the viewer was walking. Inventer la Terre could dispense with the 
screen surface altogelher and achieve an explicil oplical conjunclion of ihe 
virlual image space and ihe aclual museum space surrounding ihe viewer. 

Lei us consider now ihe opporlunilies offered by ihe ihree-dimensional 
methods of computer visualizalion. If we describe the two-dimensional digital 
image as a planar raster of numeric values (pixels), then the three- 
dimensional digital image is a volumetric rasler of numeric values (voxels). 
The viewer's eye position inside ihal volumetric space determines each 
momenlary displayed or-dering of ihal raster. Thus a three-dimensional virtual 
image space can be created which always reveais itself inleraclively in relalion 
lo the viewer. "Virlual realily" and "cyberspace" are the fashionable lerms 
which have come lo be used lo describe ihe technologies which allow a 
heightened degree of viewer "immersion" in such a virtual image space — 
technologies which can simulale the sensual co-ordinales of ihe real world in a 
new arena of ficlional visualization. 

The Legible City (Artec 89, Nagoya, Japan, 1989) was a three-dimensional 
digilal image whose virlual size was approximalely six square kilomelers. The 
viewer could interactively travel in ibis space by riding a bicycle which was stand- 
ing in front of a large video projection screen. The ground plan of this space 
was based on a number of different cities: Manhallan, Amsterdam, and 
Karlsruhe. In-slead of buildings lining the avenues and slreels, ihe wriler Dirk 
Groeneveld wrote a number of stories for this work, and ihe lelters and words 
of his text conslilule ils whole visual archileclure. In this way ihe cily was 
Iransformed inlo 
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i kind of three-dimensional book, and bicycling there was an activity of reading 
where each bicyclist, by choosing his/her own path, made a unique and 
personal reconstruction of the text. The "real-time" interaction of the bicycle and 
the computer-generated image allowed the bicyclist complete freedom to move 
any-where in this virtual image space. 

In The Legible City, the space of visualization was virtually located beyond the 
surface of the projection screen and thus outside the actual room where the 
bicyclist was situated. In Alice's Room (Kanagawa Science Park, Japan, 1989) 
and The Virtual Museum (Art Frankfurt, 1991) I located the virtual imagery inside 
the actual room where the viewers were standing. Such a digital conjunction of 
virtual and actual space evokes a mannerist ambiguity — different orders of sim- 
ulation are located in a meta-dimensional structure that mirrors a confluence of 
the real and the fictional. This location of the virtual space in a contiguous rela- 
tionship with the real space establishes a discourse in that fine zone between the 



virtual and the actual which resembles what Duchamp called "the inframince." It 
is here that I believe the most interesting and challenging opportunities for 
artistic propositions exist. 

The world is becoming increasingly museified. This tendency towards pre- 
mature conservation may be relieved by a virtual museum architecture that is as 
provisional as the culture it embodies. The Virtual Museum is a three- 
dimensional computer-generated museum containing an immaterial 
constellation of rooms and exhibits. Its apparatus is a round, rotating platform on 
which is located a large video screen, computing equipment, and a chair on 
which the viewer sits. From the chair the viewer interactively controls his/her 
movement through The Virtual Museum. Fonvard and backward movement of 
the chair causes forward and backward movement of the viewer's eye position in 
the museum space represented on the video screen. Turning the chair causes a 
rotation of this virtual image space and also a synchronous physical rotation of 
the platform itself, so that the viewer moves simultaneously in both the virtual and 
real environments. 

The architecture of The Virtual Museum is constituted by five rooms, all ot 
which reproduce the architecture of the real room in which the installation is 
located, so generating the conjunction of the real and virtual spaces. The first 
room shows a representation of the installation itself, with its platform, computer, 
video screen, and chair. Moving through The Virtual Museum the viewer in effect 
transporta himself out of this chair, and so can see the installation represented 
outside his virtual point of view. The other four rooms are consecutively entered 
just by 
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"As WeMayThink" (1945) 



VannevarBush and the differential analyzer. Courtesy ofMIT 
Computer Museum. 

"Presumably man's spirit should be elevated jfhe can better review 
his shady past and analyse more completely and objectively his 
present problems. He has built a civilization so complex that he 
needs to mechame his records more fully ifhe js to push his 
experiment to jts logical conclusion and not merely become 
bogged down part way there by overtaxing his limited memory. His 
excursions may be more enjoyable ifhe can reacquire the privilege 
of forgetting the manifold things he does not need to have 
immediately at hand, with some assurance that he can find them 
again jfthey prove important." 
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For the biologists, and particularly for the medical scientists, there can be lit- 
tle indecision, for their war has hardly required them to leave the oid paths. 
Many indeed have been able to carry on their war research in their familiar 
peace-time laboratories. Their objectives remain much the same. 
It is the physicists who have been thrown most violently off stride, who have left 
academic pursuits for the making of strange destructive gadgets, who have 
had to devise new methods for their unanticipated assignments. They have 
done their part on the devices that made it possible to turn back the enemy, 
have worked in combined effort with the physicists of our allies. They have felt 
within themselves the stir of achievement. They have been part of a great team. 
Now, as peace approaches, one asks where they will find objectives worthy of 
their best. 
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Of what lasting benefit has been man's use of science and of the new 
instrumente which his research brought into existence? First, they have 
increased his control of his material environment. They have improved his 
food, his clothing, his shel-ter; they have increased his security and released 
him partly from the bondage oi bare existence. They have given him increased 
knowledge of his own biologica] processes so that he has had a progressive 
freedom from disease and an increased span of life. They are illuminating the 
interactions of his physiological and psy-chological functions, giving the 
promise of an improved mental health. 

Science has provided the swiftest communication between individuals; it has 
provided a record of ideas and has enabled man to manipulate and to make 
ex-tracts from that record so that knowledge evolves and endures throughout 
the life of a race rather than that of an individual. 

There is a growing mountain of research. But there is increased evidence 
that we are being bogged down today as specialization extends. The 
investigator Is staggered by the findings and conclusions of thousands of other 
workers — conclusions which he cannot find time to grasp, much less to 
remember, as they ap-pear. Yet specialization becomes increasingly 
necessary for progress, and the sffort to bridge between disciplines is 
correspondingly superficial. 

Professionally our methods of transmitting and reviewing the results of re- 
iearch are generations ofd and by now are totally inadequate for their 
purpose, [f the aggregate time spent in writing scholarly works and in reading 
them could 3e evaluated, the ratio between these amounts of time might well 
be startling, rhose who conscientiously attempt to keep abreast of current 
thought, even in estricted fields, by ciose and continuous reading might well shy 
away from an ex- 

imination calculated to show how much of the previous month's efforts could 
produced on cali. Mendel's concept of the laws of genetics was lost to the wc 
For a generation because his publication did not reach the few who were capa 
of grasping and extending it; and this sort of catastrophe is undoubtedly be 
repeated all about us, as truly significan! attainments become lost in the mas 
the inconsequential. 

The difficulty seems to be, not so much that we publish unduly in view of 
extent and variety of present day interests, but rather that publication has b 
extended far beyond our present ability to make real use of the record. The si 
mation of human experience is being expanded at a prodigious rate, and 
means we use for threading through the consequent maze to the momentarily 
portant Item is the same as was used in the days of square-rigged ships. 



But there are signs of a change as new and powerful instrumentalities ce 
into use. Photocells capable of seeing things in a physical sense, advanced p 
tography which can record what is seen or even what is not, thermionic tubes 
pable of controlling potent forces under the guidance of less power tha 
mosquito uses to vibrate his wings, cathode ray tubes rendering visible an 
currence so brief that by comparison a microsecond is a long time, relay con 
nations which will carry out involved sequences of movements more reli; 
than any human operator and thousands of times as fast — there are plenty of 
chanical aids with which to effect a transformation in scientific records. 

Two centuries ago Leibnitz invented a calculating machine which emboe 
most of the essential features of recent keyboard devices, but it could not t 
come into use. The economics of the situation were against it: the labor invo! in 
constructing it, before the days of mass production, exceeded the labor t< 
saved by its use, since all it could accomplish could be duplicated by suffic 
use of pencil and paper. Moreover, it would have been subject to frequent bri 
down, so that it could not have been depended upon; for at that time and 1 
after, complexity and unreliability were synonymous. 

Babbage, even with remarkably generous support for his time, could not j 
duce his great arithmetical machine. His idea was sound enough, but const 
tion and maintenance costs were then too heavy. Had a Pharaoh been g 
detailed and explicit designs of an automobile, and had he understood 0 
completely, it would have taxed the resources of his kingdom to have fashic 
the thousands of parts for a single car, and that car would have broken dowi 
the first trip to Gi/a. 

Machines with interchangeable parts can now be constructed with great e< 
omv of effort. In snite of much comnlexitv. thev nerform reliablv. Witnes? 
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uumble typewriter, or the movie camera, or the automobile. Electrical contacts 
tiave ceased to stick when thoroughly understood. Note the automatic 
telephone sxchange, which has hundreds of thousands of such contacts, and 
yet is reliable. A spider web of metal, sealed in a thin glass container, a wire 
heated to brilliant A low, in short, the thermionic tube of radio sets, is made by 
the hundred million, tossed about in packages, plugged into sockets — and it 
works! Its gossamer parts, the precise location and alignment involved in its 
construction, would have oc-cupied a master craftsman of the guild for 
months; now it is built for thirty cents. The world has arrived at an age of cheap 
complex devices of great reliability; and something is bound to come of it. 

A record, if it is to be useful to science, must be continuously extended, it 
must be stored, and above all it must be consulted. Today we make the 
record con-ventionally by writing and photography, followed by printing; but we 
also record on film, on wax disks, and on magnetic wires. Even if utterly new 
recording pro-cedures do not appear, these present ones are certainly in the 
process of modifi-cation and extension. 

Certainly progress in photography is not going to stop. Faster material and 
lenses, more automatic cameras, finer-grained sensitive compounds to allow 
an extension of the minicamera idea, are all imminent. Let us project this 
trend ahead to a logical, if not inevitable, outcome. The camera hound of the 
future wears on his forehead a lump a little larger than a walnut. It takes 
pictures 3 mil-limeters square, later to be projected or enlarged, which after all 
involves only a factor of 10 beyond present practice. The lens is of universal 
focus, down to any distance accommodated by the unaided eye, simply 
because it is of short focal length. There is a built-in photocell on the walnut 
such as we now have on at least one camera, which automatically adjusts 
exposure for a wide range of illumina-tion. There is film in the walnut for a 
hundred exposures, and the spring for op-erating its shutter and shifting its film 
is wound once for all when the film clip is inserted. It produces its result in full 
color. It may well be stereoscopic, and record with two spaced glass eyes, 
for striking improvements in stereoscopic technique are just around the 
corner. 

The cord which trips its shutter may reach down a man's sleeve within 
easy reach of his fingers. A quick squeeze, and the picture is taken. On a pair 
of ordi-nary glasses is a square of fine lines near the top of one lens, where it is 
out of thi of ordinary vision. When an object appears in that square, it is lined 
up for its picture. As the scientist of the future moves about the laboratory or 



the field, svery time he looks at something worthy of the record, he trips the 
shutter and in it goes, without even an audible click. Is this all fantastic? The 
only fantastic thing about it is the idea of making as many pictures as would 
result from its use. 
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Will there be dry photography? It is already here in two forras. When Brady 
made his Civil War pictures, the piate had to be wet at the time of exposure. 
Now it has to be wet during development instead. In the future perhaps it 
need not be wetted at all. There have long been films impregnated with diazo 
dyes which form a picture without development, so that it is already there as 
soon as the camera has been operated. An exposure to ammonia gas 
destroys the unex-posed dye, and the picture can then be taken out into the 
light and examined. The process is now slow, but someone may speed it up, 
and it has no grain diificul-ties such as now keep photographic researchers 
busy. Often it would be advan-tageous to be able to snap the camera and to 
look at the picture immediately. 

Another process now in use is also slow, and more or less clumsy. For fifty 
years impregnated papers have been used which turn dark at every point 
where an electrical contact touches them, by reason of the chemical change 
thus pro-duced in an iodine compound included in the papen They have 
been used to make records, for a pointer moving across them can leave a 
trail behind. If the electrical potential on the pointer is varied as it moves, the 
line becomes light or dark in accordance with the potential. 

This scheme is now used in facsimile transmission. The pointer draws a 
set of closely spaced lines across the paper one after another. As it moves, its 
potential is varied in accordance with a varying current received over wires from 
a dis-tant station, where these variations are produced by a photocell which is 
similarly scanning a picture. At every instant the darkness of the line being 
drawn is made equal to the darkness of the point on the picture being 
observed by the photocell. Thus, when the whole picture has been covered, a 
replica appears at the re-ceiving end. 

A scene itself can be just as well looked over line by line by the photocell in 
this way as can a photograph of the scene. This whole apparatus constitutes 
a camera, with the added feature, which can be dispensed with if desired, of 
making its picture at a distance. It is slow, and the picture is poor in detail. Still, it 
does give another process of dry photography, in which the picture is finished 
as soon as it is taken. 

It would be a brave man who would predict that such a process will always 
remain clumsv. slow, and faultv in detail. Television eauioment todav transmiti 
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sixteen reasonably good pictures a second, and it involves only two essential 
dif-ferences from the process described above. For one, the record is made 
by a moving beam of electrons rather than a moving pointer, for the reason 
that an electron beam can sweep across the picture very rapidly indeed. The 
other dif-ference involves merely the use of a screen which glows momentarily 
when the electrons hit, rather than a chemically treated paper or film which is 
permanently altered. This speed is necessary in television, for motion 
pictures rather than stills are the object. 

Use chemically treated film in place of the glowing screen, allow the appara- 
tus to transmit one picture only rather than a succession, and a rapid camera 
for dry photography results. The treated film needs to be far faster in action than 
pre-sent examples, but it probably could be. More serious is the objection 
that this scheme would involve putting the film inside a vacuum chamber, for 
electron beams behave normally only in such a rarefied environment. This 
difficulty could be avoided by allowing the electron beam to play on one side of 
a partition, and by pressing the film against the other side, if this partition were 
such as to allow the electrons to go through perpendicular to its surface, and 
to prevent them from spreading out sideways. Such partitions, in crude form, 
could certainly be constructed, and they will hardly hold up the general 
development. 

Like dry photography, microphotography still has a long way to go. The 
basic scheme of reducing the size of the record, and examining it by 
projection rather than directly, has possibilities too great to be ignored. The 
combination of optical projection and photographic reduction is already 
producing some results in microfilm for scholarly purposes, and the 
potentialities are highly sug-gestive. Today, with microfilm, reductions by a 
linear factor of 20 can be employed and still produce full clarity when the 
material is re-enlarged for ex-amination. The limits are set by the graininess of 
the film, the excellence of the optical system, and the efficiency of the light 
sources employed. All of these are rapidly improvmg. 

Assume a linear ratio of 100 for future use. Consider film of the same thick- 
ness as paper, although thinner film will certainly be usable. Even under 
these conditions there would be a total factor of 10,000 between the bulk of 
the ordi-nary record on books and its microfilm replica. 
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The Encyclopaedia Britannica could be reduced to the volume of a 
matchbox. A library of a million volumes could be compressed into one end of 
a desk. If the human race has produced since the invention of movable type 
a total record, in the form of magazines, newspapers, books, tracts, 
advertising blurbs, correspondence, having a volume corresponding to a 
billion books, the whole affair, assembled and compressed, 
could be lugged offin a moving van. Mere compression, of course, is not 
enougl one needs not only to make and store a record but also be able to 
consult it, and this aspect of the matter comes laten Even the modern great 
library is not gener-ally consulted; it is nibbled at by a few. 

Compression is important, however, when it comes to costs. The material for 
the microfilm Britannica would cost a nickel, and it could be mailed anywhere 
for a cent. What would it cost to print a million copies? To print a sheet of news- 
paper, in a large edition, costs a small fraction of a cent. The entire material of 
the Britannica in reduced microfilm form would go on a sheet eight and one-half 
by eleven inches. Once it is available, with the photographic reproduction 
mediods of the future, duplicates in large quantities could probably be turned 
out for a cent apiece beyond the cost of materials. The preparation of the 
original copy? That introduces the next aspect of the subject. 

To make the record, we now push a pencil or tap a typewriter. Then comes 
the process of digestion and correction, followed by an intricate process of 
typeset-ting, printing, and distribution. To consider the first stage of the 
procedure, will the author of the future cease writing by hand or typewriter and 
talk directly to the record? He does so indirectly, by talking to a stenographer or 
a wax cylinder: but the elements are all present if he wishes to have his talk 
directly produce a typed record. All he needs to do is to take advantage of 
existing mechanisms and to alter his language. 

At a recent World Fair a machine called a Voder was shown. A girl. stroked its 
keys and it emitted recognizable speech. No human vocal chords entered into 
the procedure at any point; the keys simply combined some electrically 
produced vi-brations and passed these on to a loud-speaker. In the Bell 
Laboratories there is the converse of this machine, called a Vocoder. The 
loudspeaker is replaced by a microphone, which picks up sound. Speak to it, 
and the corresponding keys move. This may be one element of the 
postulated system. 



The other element is found in the stenotype, that somewhat disconcerting de- 
vice encountered usually at public meetings. A girl strokes its keys languidly 
and looks about the room and sometimes at the speaker with a disquieting 
gaze. From it emerges a typed strip which records in a phonetically 
simplified language a record of what the speaker is supposed to have said. 
Later this strip is re-typed into ordinary language, for in its nascent form it is 
intelligible only to the 
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initiated. Combine these two elements, let the Vocoder run the stenotype, and 
the result is a machine which types when talked to. 

Our present languages are not especially adapted to this sort of 
mechaniza-tion, it is true. It is strange that the inventors of universal 
languages have not seized upon die idea of producing one which better fitted 
the technique for trans-tnitting and recording speech. Mechanization may yet 
force the issue, especially jn the scientific field; whereupon scientific jargon 
would become still less intel-iigible to the layman. 

One can now picture a fiiture investigator in his laboratory. His hands are free, 
md he is not anchored. As he moves about and observes, he photographs 
and :omments. Time is automatically recorded to tie the two records together. 
If he goes into the field, he may be connected by radio to his recorder. As he 
pondera Dver his notes in the evening, he again talks his comments into the 
record. His typed record, as well as his photographs, may both be in 
miniature, so that he projects them for examination. 

Much needs to occur, however, between the collection of data and 
observa-tions, the extraction of parallel material from the existing record, and 
the final in-sertion of new material into the general body of the common record. 
For mature thought there is no mechanical substitute. But creative thought 
and essentially repetitive thought are very different things. For the latter there 
are, and may be. powerful mechanical aids. 

Adding a column of figures is a repetitive thought process, and it was long ago 
properly relegated to the machine. True, the machine is sometimes controlled 
by i keyboard, and thought of a sort enters in reading the figures and poking the 
cor-responding keys, but even this is avoidable. Machines have been made 
which will read typed figures by photocells and then depress the 
corresponding keys; these ire combinations of photocells for scanning the 
type, electric circuits for sorting the consequent variations, and relay circuits for 
interpreting the result into the ac-tion of solenoids to pull the keys down. 



All this complication is needed because of the clumsy way in which we have 
learned to write figures. If we recorded them positionally, simply by the config- 
uration of a set of dots on a card, the automatic reading mechanism would 
become comparatively simple. In fact if the dots are boles, we have the ) 
unched-card machine long ago produced by Hollorith for the purposes of the 
•ensus, and now used throughout business. Some types of complex 
businesses ould hardly operate without these machines. 

Adding is only one operation. To perform arithmetical computation involves 
iso subtraction, multiplication, and division, and in addition snmp tnptbnH fnr 
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temporary storage of results, removal from storage for further manipulation, 
a« recording of final results by printing. Machines for these purposes are now of 
tw« types: keyboard machines for accounting and the like, manually controlled 
for tr.e insertion of data, and usually automatically controlled as far as the 
sequence o operations is concerned; and punched-card machines in which 
separate oper tions are usually delegated to a series of machines, and the 
cards then transferret bodily from one to another. Both forms are very useful; 
but as far as complex < putations are concerned, both are still in embryo. 

Rapid electrical counting appeared soon after the physicists found it desir 
to count cosmic rays. For their own purposes the physicists promptly constructi 
thermionic-tube equipment capable of counting electrical impulses at the 
rate« 100,000 a second. The advanced arithmetical machines of the future 
wul electrical in nature, and they will perform at 100 times present speeds, or 
me 

Moreover, they will be far more versatile than present commercial macr so 
that they may readily be adapted for a wide variety of operations. They will 1 
controlled by a control card or film, they will select their own data and manij: 
late it in accordance with the instructions thus inserted, they will perform < 
plex arithmetical computations at exceedingly high speeds, and they wul i 
results in such form as to be readily available for distribution or for later i 
manipulation. Such machines will have enormous appetites. One of their, 
take instructions and data from a whole roomful of girls armed with simple 1 
board A punches, and will deliver sheets of computed results every few mi ir- 
rhere will always be plenty of things to compute in the detailed affairs of 
millicaf people doing complicated things. 

The repetitive processes of thought are not confined, however, to 
matters arithmetic and statistics. In fact, every time one combines and records 
facts in ac cordance with established logical processes, the creative aspect 
of thinking . concerned only with the selection of the data and the process to 



be employed an the manipulation thereafter is repetitive in nature and henee 
a fit matter to be egated to the machine. Not so much has been done along 
these lines, beyond bounds of arithmetic, as might be done, primarily 
because of the economics the situation. The needs of business and the 
extensive market obviously waiti assured the advent of mass-produced 
arithmetical machines just as soon as duction methods were sufficiently 
advanced. 
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With machines for advanced analysis no such situation existed; for there 
was and is no extensive market; the users of advanced methods of 
manipulating data ire a very small part of the population. There are, however, 
machines for solving differential equations — and functional and integral 
equations, for that matter. There are many special machines, such as the 
harmonic synthesizer which pre-dicts the tides. There will be many more, 
appearing certainly first in the hands of the scientist and in small numbers. 

If scientific reasoning were limited to the logical processes of arithmetic, we 
should not get far in our understanding of the physical world. One niight as well 
attempt to grasp the game of poker entirely by the use of the mathematics of 
probability. The abacus, with its beads strung on parallel wires, led the Arabs 
to positional numeration and the concept of zero many centuries before the 
rest of the world; and it was a useful tool — so useful that it still exists. 

It is a far cry from the abacus to the modern keyboard accounting machine. 
U will be an equal step to the arithmetical machine of the future. But even this 
new machine will not take the scientist where he needs to go. Relief must be 
secured from laborious detailed manipulation of higher mathematics as well, if 
the users of it are to free their brains for something more than repetitive 
detailed transfor-mations in accordance with established rules. A 
mathematician is not a man who can readily manipulate figures; often he 
cannot. He is not even a man who can readily perform the transformations of 
equations by the use of calculus. He is pri-marily an individual who is skilled in 
the use of symbolic logic on a high plane, and especially he is a man of 
intuitive judgment in the choice of the manipulative processes he employs. 

AU else he should be able to turn over to his mechanism, just as 
confidently as he turns over the propelling of his car to the intricate 
mechanism under the hood. Only theri will mathematics be practically 
effective in bringing the grow-ing knowledge of atomistics to the useful 



solution of the advanced problems of chemistry, metallurgy, and biology. For 
this reason there still come more machines to handle advanced mathematics 
for the scientist. Some of them will be sufficiently bizarre to suit the most 
fastidious connoisseur of the present arti-facts of civilization. 

The scientist, however, is not the only person who manipulates data and 
examines the world about him by the use of logical processes, although he 
sometimes 
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preserves this appearance by adopting into the fold anyone who becomes logi 
cal, much in the manner in which a British labor leader is elevated to knighthood 
Whenever logical processes of thought are employed — that is, whenever though 
for a time runs along an accepted groove — there is an opportunity for the ma 
chine. Formal logic used to be a keen instrument in the hands of the teacher u 
bis trying of students' souls. It is readily possible to construct a machine whid will 
manipulate premises in accordance with formal logic, simply by the eleve use of 
relay circuits. Put a set of premises into such a device and turn the cranl and it 
will readily pass out conclusion after conclusion, all in accordance with lo{| ical law, 
and with no more slips than would be expected of a keyboard adding nu chine. 

Logic can become enormously difficult, and it would undoubtedly be well t 
produce more assurance in its use. The machines for higher analysis have usi 
ally been equation solvers. Ideas are beginning to appear for equation tram 
formers, which will rearrange the relationship expressed by an equation i 
accordance with strict and rather advanced logic. Progress is inhibited by the a 
ceedingly crude way in which mathematicians express their relationships. The 
employ a symbolism which grew like Topsy and has little consistency; a stranj fact 
in that most logical field. 

A new symbolism, probably positional, must apparently precede the redui tion 
of mathematical transformations to machine processes. Then, on beyond ti strict 
logic of the mathematician, lies the application of logic in everyday affair We may 
some day click off arguments on a machine with the same assuran» that we now 
enter sales on a cash register. But the machine of logic will not lo( like a cash 
register, even of the streamlined model. 

So much for the manipulation of ideas and their insertion into the recor Thus 
far we seem to be worse off than befo re — for we can enormously extend d record; 
yet even in its present bulk we can hardly consult it. This is a much larg matter 
than merely the extraction of data for the purposes of scientific researc it involves 
the entire process by which man profits by his inheritance of acquin knowledge. 
The prime action of use is selection, and here we are halting indee There may be 



millions of fine thoughts, and the account of the experience ( which they are 
based, all encased within stone walls of acceptable architects form; but if the 
scholar can get at only one a week by diligent search, his syntt ses are not likely 
to keep up with the current scene. 

Selection, in this broad sense, is a stone adze in the hands of a cabinetmak 
Yet, in a narrow sense and in other areas, something has already been done ra 
chanicallv on selection. The nersonnel officer of a factorv droos a stack of a fi 
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thousand employee cards into a selecting machine, sets a code in 
accordance with an established convention, and produces in a short time a 
list of all em-ployees who live in Trenton and know Spanish. Even such 
devices are much too slow when it comes, for example, to matching a set of 
fingerprints with one of five million on file. Selection devices of this sort will 
soon be speeded up from their present rate of reviewing data at a few hundred 
a minute. By the use of photocells and microfilm they will survey items at the 
rate of a thousand a second, and will print out duplicates of those selected. 

This process, however, is simple selection: it proceeds by examining in turn 
every one of a large set of items, and by picking out those which have certain 
spec-ified characteristics. There is another form of selection best illustrated by 
the au-tomatic telephone exchange. You dial a number and the machine 
selects and connects just one of a million possible stations. It does not run 
over them all. It pays attention only to a class given by a first digit, then only to 
a subclass of this given by the second digit, and so on; and thus proceeds 
rapidly and almost un-erringly to the selected station. It requires a few 
seconds to make the selection, although the process could be speeded up if 
increased speed were economically warranted. If necessary, it could be 
made extremely fast by substituting thermionic-tube switching for 
mechanical switching, so that the full selection could be made in one one- 
hundredth of a second. No one would wish to spend the money necessary to 
make this change in the telephone system, but the general idea is applicable 
elsewhere. 

Take the prosaic problem of the great department store. Every time a 
charge sale is made, there are a number of things to be done. The inventory 



needs to be revised, the salesman needs to be given credit for the sale, the 
general accounts need an entry, and, most important, the customer needs to 
be charged. A central records device has been developed in which much of 
this work is done conve-niently. The salesman places on a stand the 
customer's identification card, his own card, and the card taken from the 
article sold — all punched cards. When he pulis a lever, contacts are made 
through the holes, machinery at a central poinl makes the necessary 
computations and entries, and the proper receipt is printed for the salesman 
to pass to the customer. 

But there may be ten thousand charge customers doing business with 
the store, and before the full operation can be completed someone has to 
select the ight card and inseri it at the central office. Now rapid selection 
can slide jusl he proper card into position in an instant or two, and return it 
afterward. An->ther difficulty occurs, however. Someone must read a total on 
the card, so thai he machine can add its computed item to it. Conceivably the 
cards might be 01 
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the dry photography type I have described. Existing totals could then be read 
by photocell, and the new total entered by an electron beam. 

The cards may be in miniature, so that they occupy little space. They must 
move quickly. They need not be transferred far, but merely into position so that 
die photocell and recorder can operate on them. Positional dots can enter the 
data. At the end of the month a machine can readily be made to read these and 
to print an ordinary bill. With tube selection, in which no mechanical parts are 
involved in the switches, little time need be occupied in bringing the correct 
card into use — a second should suffice for the entire operation. The whole record 
on the card may be made by magnetic dots on a steel sheet if desired, instead of 
dots to be observad optically, following the scheme by which Poulsen long ago 
put speech on a magnetic wire. This method has the advantage of simplicity and 
ease of erasure. By using photography, however, one can arrange to project the 
record in enlarged form and at a distance by using the process common in 
television equipment. 

One can consider rapid selection of this form, and distant projection foi 
other purposes. To be able to key one sheet of a million before an operator in a 
second or two, with the possibility of then adding notes thereto, is suggestive in 
many ways. It might even be of use in libraries, but that is another story. At any 
rate, there are now some interesting combinations possible. One might, for ex- 
ample, speak to a microphone, in the manner described in connection with the 
speech-controlled typewriter, and thus make his selections. It would certainlj 
beat the usual file clerk. 



6 



The real heart of the matter of selection, however, goes deeper dian a lag in the 
adoption of mechanisms by libraries, or a lack of development of devices foi 
their use. Our ineptitude in getting at the record is largely caused by the artifi- 
ciality of systems of indexing. When data of any sort are placed in storage, they 
are filed alphabetically or numerically, and information is found (when it is) by 
tracing it down from subclass to subclass. It can be in only one place, unless du- 
plicates are used; one has to have rules as to which path will locate it, and the rules 
are cumbersome. Having found one Item, moreover, one has to emerge from the 
system and re-enter on a new path. 

The human mind does not work that way. It operales by association. With 
one Item in its grasp, it snaps instantly to the next that is suggested by the asso 
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ciation of thoughts, in accordance with some intricate web of trails carried by 
the cells of the brain. It has other characteristics, of course; trails that are not 
fre-quently followed are prone to fade, Items are not fully permanent, 
memory is transitory. Yet the speed of action, the intricacy of trails, the detail of 
mental pic-tures, is awe-inspiring beyond all else in nature. 

Man cannot hope fully to duplicate this mental process artificially, but he 
certainly ought to be able to learn from it. In minor ways he may even improve, 
for his records have relative permanency. The first idea, however, to be drawn 
from the analogy concerns selection. Selection by association, rather than 
in-dexing, may yet be mechanized. One cannot hope thus to equal the 
speed and flexibility with which the mind follows an associative trail, but it 
should be pos-sible to beat the mind decisively in regard to the permanence 
and clarity of the Items resurrected from storage. 

Consider a future device for individual use, which is a sort of mechanized 
private file and library. It needs a name, and, to coin one at random, "memex" 
wul do. A memex is a device in which an individual stores all his books, 
records, and Communications, and which is mechanized so that it may be 
consulted with ex-ceeding speed and flexibility. It is an enlarged intimate 
supplement to his memory. 

It consists of a desk, and while it can presumably be operated from a 
distance, it is primarily the piece of furniture at which he works. On the top 
are slanting translucent screens, on which material can be projected for 
convenient reading. There is a keyboard, and sets of buttons and levers. 
Otherwise it looks like an or-dinary desk. 



In one end is the stored material. The matter of bulk is well taken care of by 
improved microfilm. Only a small part of the interior of the memex is devoted to 
storage, the rest to mechanism. Yet if the user inserted 5,000 pages of 
material a day it would take him hundreds of years to fill the repository, so he 
can be profligate and enter material freely. 

Most of the memex contents are purchased on microfilm ready for 
insertion. Books of all sorts, pictures, current periodicals, newspapers, are 
thus obtained and dropped into place. Business correspondence takes the 
same path. And there is provision for direct entry. On the top of the memex 
is a transparent platen. On this are placed longhand notes, photographs, 
memoranda, all sorts of things. When one is in place, the depression of a 
lever causes it to be pho-ographed onto the next blank space in a section 
of the memex film, dry plio-.ography being employed. 

There is, of course, provision for consultation of the record bv the usual 
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scheme of indexing. If the user wishes to consult a certain book, he taps its 
code an the keyboard, and the title page of the book promptly appears before 
him, projected onto one of his viewing positions. Frequently used codes are 
mnemonic, so that he seldom consults his code book; but when he does, a 
single tap of a key projects it for his use. Moreover, he has supplemental 
levers. On deflecting one of these levers to the right he runs through the book 
before him, each page in turn being projected at a speed which just allows a 
recognizing glance at each. If he deflects it further to the right, he steps 
through the book 10 pages at a time; still further at 100 pages at a time. 
Deflection to the left gives him the same control backwards. 

A special button transfers him immediately to the first page of the Index. Any 
given book of his library can thus be called up and consulted with far greater fa- 
cility than if it were taken from a shelf. As he has several projection positions, he 
can leave one Item in position while he calis up another. He can add marginal 
notes and comments, taking advantage of one possible type of dry 
photography and it could even be arranged so that he can do this by a stylus 
scheme, such aa is now employed in the telautograph seen in railroad 
waiting rooms, just as though he had the physical page before him. 



All this is conventional, except for the projection forward of present-day mech 
anisms and gadgetry. It affords an immediate step, however, to associative in 
dexing, the basic idea of which is a provision whereby any Item may be 



causee at will to select immediately and automatically another. This is the 
essential fea ture of the memex. The process of tying two iteras together is 
the importan thing. 

When the user is building a trail, he names it, inseris the name in his cod< 
book, and taps it out on his keyboard. Before him are the two Items to be joined 
projected onto adjacent viewing positions. At the bottom of each there are j 
number of blank code spaces, and a pointer is set to indicate one of these on 
eacl itera. The user taps a single key, and the Items are permanently joined. 
In eacl code space appears the code word. Out of view, but also in the code 
space, is in serted a set of dots for photocell viewing; and on each item these 
dots by their po sitions designate the Index number of the other Item. 

Thereafter, at any time, when one of these items is in view, the other can b 
mstanrlv recalled merelv bv taDDIne: a button below the corresponding cod 
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space. Moreover, when numerous Items have been thus joined together to 
form i trail, they can be reviewed in turn, rapidly or slowly, by deflecting a lever 
like that used for turning the pages of a book. It is exactly as though the physical 
Items bad been gathered together from widely separated sources and bound 
together to form a new book. It is more than this, for any Item can bejoined 
into numerous trails. 

The owner of the memex, let us say, is interested in the origin and properties 
of the bow and arrow. Specifically he is studying why the short Turkish bow was 
apparently superior to the English long bow in the skirmishes of the Crusades. 
He has dozens of possibly pertinent books and articles in his memex. First 
he runs through an encyclopedia, finds an interesting but sketchy article, 
leaves it projected. Next, in a history, he finds another pertinent Item, and ties 
the two together. Thus he goes, building a trail of many Items. Occasionally 
he inseris a comment of his own, either linking it into the main trail or joining it 
by a side trail to a particular item. When it becomes evident that the elastic 
properties of avail-able materials had a great deal to do with the bow, he 
branches off on a side trail which takes him through textbooks on elasticity and 
tables of physical constants. He inseris a page of longhand analysis of his 
own. Thus he builds a trail of his interest through the maze of materials 
available to him. 

And his Irails do not fade. Several years later, his talk with a friend turns to 
the queer ways in which a people resist innovations, even of vital interest. He 
has an example, in the fact that the outraged Europeans still failed to adopt the 
Turkish bow. In fact he has a trail on it. A touch brings up the code book. 
Tapping a few keys projects the head of the trail. A lever runs through it at will, 



stopping at interesting Items, going off on side excursions. It is an interesting 
trail, pertinent to the discussion. So he sets a reproducer in action, 
photographs the whole trail out, and passes it to his friend for insertion in his 
own memex, there to be linked into the more general trail. 
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Wholly new forms of encyclopedias will appear, ready made with a mesh of 
as-sociative trails running through them, ready to be dropped into the memex 
and there amplified. The lawyer has at his touch the associated opinions and 
deci-sions of his whole experience, and of the experience of friends and 
authorities. The patent attorney has on cali the millions of issued patents, with 
familiar trails to everv ooint of his client's interest. The nhvsician. nuzzled bv a 
natient's reac- 
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tions, strikes the trail established in studying an earlier similar case, and runs 
rapidly through analogous case histories, with side references to the classics 
for the pertinent anatomy and histology. The chemist, struggling with the 
synthesis oian organic compound, has all the chemical literature before him 
in his labo-ratory, with trails following the analogies of compounds, and side 
trails to their physical and chemical behavior. 

The historian, with a vast chronological account of a people, parallels it with 
a skip trail which stops only on the salient Items, and can follow at any time 
con-temporary trails which lead him all over civilization at a particular epoch. 
There is a new profession of trail blazers, those who find delight in the task of 
estab-lishing useful trails through the enormous mass of the common record. 
The in-heritance from the master becomes not only his additions to the 
world's record, but for his disciples the entire scaffolding by which they were 
erected. 

Thus science may implement the ways in which man produces, stores, 
and consults the record of the race. It might be striking to outline the 
instrumentali-ties of the future more spectacularly, rather than to stick closely 
to methods and elements now known and undergoing rapid development, as 
has been done here. Technical difficulties of all sorts have been ignored, 
certainly, but also ignored are means as yet unknown which may come any day 
to accelerate technical progress as violently as did the advent of the thermionic 
tube. In order that the picture ma> not be too commonplace, by reason of 
sticking to present-day patterns, it may be well to mention one such possibility, 



not to prophecy but merely to suggest, foi prophecy based on extension of 
the known has substance, while prophec) founded on the unknown is only a 
doubly involved guess. All our steps in creat-ing or absorbing material of the 
record proceed through one of the senses — thi tactile when we touch keys, 
the oral when we speak or listen, the visual when wt read. Is it not possible that 
some day the path may be established more direcdy: 

We know that when the eye sees, all the consequent information is 
transmit-ted to die brain by means of electrical vibrations in the channel of the 
optic nerve. This is an exact analogy with the electrical vibrations which occur 
in the cable of a television set: they convey the picture from the photocells 
which see it to the radio transmitter from which it is broadcast. We know further 
that if we can ap-proach that cable with the proper instruments, we do not need 
to touch it; we can pick up those vibrations by electrical induction and thus 
discover and reproduce the scene which is being transmittedjust as a 
telephone wire may be tapped foi its message. 

The impulses which flow in the arm nerves of a typist convey to her fingers 
the translated information which reaches her eye or ear, in order that the 
fingers 
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may be caused to strike the proper keys. Might not these currents be 
intercepted, either in the original form in which Information is conveyed to the 
brain, or in the marvelously metamorphosed form in which they then proceed 
to the hand? 

By bone conduction we already introduce sounds: into the nerve channels 
of the deaf in order that they may hear. Is it not possible that we may learn to 
introduce them without the present cumbersomeness of first transiorming 
electri-cal vibrations to mechanical ones, which the human mechanism 
promptly transforms back to the electrical form? With a couple of electrodes 
on the skull the encephalograph now produces pen-and-ink traces which bear 
some relation to the electrical phenomena going on in the brain itself. True, 
the record is un-intelligible, except as it points out cei tain gross 
misfunctioning of the cerebral mechanism; but who would now place bounds 
on where such a thing may lead? 

In the outside world, all forms of intelligence, whether of sound or sight, 
have been reduced to the form of varying currents in an electric circuit in 
order that they may be transmitted. Inside the human frame exactly the same 
sort of process occurs. Must we always transform to mechanical movements 
in order to proceed from one electrical phenomenon to another? It is a 



suggestive thought, but it hardly warrants prediction without losing touch with 
reality and immedi-ateness. 

Presumably man's spirit should be elevated if he can better review his 
shady past and analyze more completely and objectively his present 
problems. He has built a civilization so complex that he needs to mechanize 
his records more fully if he is to push his experiment to its logical conclusion 
and not merely become bogged down part way there by overtaxing his limited 
memory. His excursions may be more enjoyable if he can reacquire the 
privilege of forgetting the manifold things he does not need to have 
immediately at hand, with some assurance that he can find them again if they 
prove important. 

The applications of science have built man a well-supplied house, and are 
teaching him to live healthily therein. They have enabled him to throw masses 
o) people against one another with cruel weapons. They may yet allow him 
truly to encompass the great record and to grow in the wisdom of race 
experience. He may perish in conflict before he learns to wield that record for 
his true good. Yet. in the application of science to the needs and desires of 
man, it would seem to be a singularly unfortunate stage at which to terminate 
the process, or to lose hope as to the outcome. 
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Ted Nelson, Computer Lib/Dream Machines. © TedNelson. Courtesy 
ofTedNelson. 

"Everything Is deeply jntertw'mgled. 
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« Ted Nelson describes himself as a person who invents paradigms and then 
makes up words to express them. Nelson had little formal training in computer 
science. However, as a graduate student in philosophy in the late 1950s and 
sarly 1960s, Nelson had two critical intellectual encounters that led him to be- 
come one of the most influential figures jn computing. One was with Vannevar 
Bush's article "As We May Think," which convinced Nelson that emerging in- 
formation technologies could extend the power of human memory. The second 
was with Samuel Taylor Coleridge's poem "Xanadu," "a magic place of literary 
memory/' in Nelson's words, that provided him with the image of a vast store- 
house of memories, and which served as the inspiration for his life's work. From 
these influences, Nelson began his quest to build creative tools that would trans- 
form the way we read and write, and jn 1963 he coined the words ftypertexi and 
hypermedia to describe the new paradigms that these tools would make possible. 

In 1974, he self-published his landmark tome on personal computing, 
Computer Lib/Dream Machines, which instantly drew an underground following of 
computer hackers, media theorists, and experimental artists. The oversized 
volume had a playful, do-it-yourself quality, with typesetting done by typewriter, 
distinc-tive hand-drawn illustrations, and a layout that suggested the nonlinear 
nature of hypermedia. Written with the author's trademark droll humor, the book 
js a stream-of-consciousness, randomly generated sequence of bombastic truths 
and speculations, a patchworklike Journal that covers everything from hardware 
sys-tems and new art forms to the decline of western civilization. Nelson was 
par-ticularly concerned with the complex nature of the creative impulse, and he 
saw the computer as the tool that would make explicit the interdependence of 
ideas, drawing out connections between literature, art music, and science, since, 
as he put jt, everything js "deeply mtertwingled." 

Nelson's critical breakthrough was to cali for a system of nonsequential 
writing that would allow the reader to aggregate meaning in snippets ("lexias," 
as Roland Barthes defined them), in the order of his or her choosing, rather than 
according to a preestablished structure fixed by the author. Nelson's vision for a 
new form of interactive literature was a hybrid concept that married Vannevar 
Bush's notion of a memory bank organized around associative links with Doug- 
las Engelbart's research jn human-computer interface and the augmentaron of 
human intellect. It was from this conceptual framework that he drew his ambi- 
tious plans for a global repository of cultural information, Xanadu. Intended to 
become the ideal new publishing system for the digital age, this visionary design 
was later eclipsed by Tim Berners-Lee's invention of the World Wide Web jn 

1989. » 
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the tissue of thought 

Uneducated people typically think of education as the learning of a lot of facts 
and skills. While facts and skills certainly have their merits, "higher education" 
is also largely concerned with tying ideas together, and especially alternative 
structures of such tying-together: with showing you the vast uncertainties of 
things. 

A wonderfuljapanese film of the fifties was called Rasho-Mon. It depicted a 
specific event — a rape — as told by five different people. As the audience 
watches the five separate stories, they must try to judge what really 
happened. 

The Rasho-Mon Principie: everything is like that. The complete truth about 
something is never kiiown. 

Nobody tells the complete truth, though some try. Nobody knows the 
complete truth. Nowhere may we find printed the complete truth. There are 
only dif-ferent views, assertions, supposed facts that support one view or 
another but are disputed by disbelievers in the particular views; and so on. 
There are "agreed-on facts," but their meaning is often in doubt. 

The great compromise of the western world is that we go by the rule: 
assume that we never know the final truth about anything, There are 
continuing Issues, Mysteries, Continuing Dialogues. What about flying 
saucers, "why Rome fell," was there a Passover Plot, and Did Roosevelt 
know Pearl Harbor would be at-tacked? 

Outsiders find the intellectual world pompous, vague in its undecided is- 
sues, stuffy in its quotes and citations. But in a way these are the sounds of 
bat-de. The clash of theories is what many find exhilarating about the 
intellectual world. The Scholarly Arena is simply a Circus Maximus in which 
these battles are scheduled. 

Many people think "science" is free from all this. These are people who do 
not know much about science. More and more is scientifically known, true; 
but it is repeatedly discovered that some scientific "knowledge" is wntrue, and 
this problem is built into the system. The important thing about science is not 
that everything will be known, or that everything unanimously believed by 
scientists is necessarily true, but that science contains a system for seeking 
untruth and purging it. 

This is the great tradition of western civilization. The Western World is, in an 
imnnrtant sense, a continuing dialogue among people who have thought 
differ- 
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ent things. "Scholarship" is the tradidon of tiying to improve, collate and re- 
solve uncertainties. The fundamental ground rules are that no issue is 
evei closed, no interconnection is impossible. It all comes down to what is 
written, because the thoughts and minds themselves, of course, do not last. 
(The appa-ratus of citation and footnote are simply a combination of hat- 
tipping, go-look-if-you-don't-believe-me, andyou-might-want-to-read-this- 
yourself.) 

"Knowledge," then — and indeed most of our civilization and what remains oi 
those previous — is a vasty cross-tangle of ideas and evidential materials, not 
a pyramid of truth. So that preserving its structure, and improving its 
accessibil-ity, is important to us all. 
Which is one reason we need hypertexts and thinkertoys. 

Psychology 
Philosophy 

Movie-making Whatevet- 

Presentational Sequences Are Arbitrary 
Hierarchies Are Typically Spurious 

| LANGUAGE 
TRUTH 

Boundaries of Fields Are Arbitrary 

Sociology 

Compartmentalized and Stratified Teaching Produces Compartmentalized and 
Stratified Minds. 



the heritage 

The past is like the receding view out the back of an automobile: the most 
recent is more conspicuous, and everything seems eventually to be lost. 

We know we could save things, but what? Those with the Job of saving 
things — the libraries and museums — save so many of the wrong things, the 



fash-lonable and expensive and high-toned things esteemed by a given time, 
and most if the rest slips past. Each generation seems to ridicule the things 
held in esteem 
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by times before, but of course this can never be a guide to what should be 
saved. And there is so much to save: music, writing, sinking Venice, vanishing 
species. 

But why should things be saved? Everything is deeply intertwingled. We save 
for knowledge and nostalgia, but what we thought was knowledge often turns 
to nostalgia, and nostalgia often brings us deeper insights that cut across our 
lives and very selves. 

Computers offer an interesting daydream: that we may be able to store 
things digitally instead ofpkysically. In other words, turn the libraries to digital 
storage; digitize paintings and photographs; even digitize the genetic codes of 
animais, so that species can be restored at future dates. 

Digital storage possesses several special advantages. Digitally stored 
materials may be copied by automatic means; corrective measures are 
possible, to prevent errors from creeping in — i.e., "no deterioration" in 
principle; and they could be kept in various places, lessening mankind's 
dependence on its eggs being all in one basket (like the Library at 
Alexandria, whose burning during the occupation of Julius Caesar was one 
of the greatest losses in human history). 

But this would of course require far more compact and reliable forms of 
digital storage than exist right now. 

Nevertheless, we better start thinking about it. Those who fear a coming 
holocaust had best think about pulling some part of mankind through, with 
some part of what he used to have. 



hypermedia 

In recent years a very basic change has occurred in presentational systems of 
all kinds. We may summarize it under the name branching, although there are 
many variants. Essentially, today's systems for presenting pictures, texts and 
whatnot can bring you different things automatically depending on what you 
do. Selec-tion of this type is generally called branching. (I have suggested the 
generic term hypermedia for presentational media which perform in this (and 
other) multidi-mensional ways.) 

A number of branching media exist or are possible. 
Branching movies or hyperfilms (see nearby). 
Branching texts or hypertexts (see nearby). 
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Branching audio, music, etc. 

Branching slide-shows. 

Wish we could get into some of that stuff here. 



hypertext 

By "hypertext" I mean non-sequential writing. 

Ordinary writing is sequential for two reasons. First, it grew out of speech and 
speech-making, which have to be sequential; and second, because books are not 
convenient to read except in a sequence. 

But the structures of ideas are not sequential. They tie together every which- 
way. And when we write, we are always trying to tie things together in non- 
sequential ways. Thefootnote is a break from sequence; but it cannot really be 
extended (though some, like Will Cuppy, have toyed with the technique). 

I have run into perhaps a dozen people who understood this instantly when I 
talked to them about it. Most people, however, act more bemused, thinking Pin 
trying to tell them something technical or pointlessly philosophical. It's not 
pointless at all: the point is, writers do better if they don't have to write in 
sequence (but may create multiple structures, branches and alternatives), and 
read-ers do better if they don't have to read in sequence, but may establish 
impressions, jump around, and try different pathways until they find the ones 
they want to study most closely. 

(The astute reader, arid anybody who's gotten to this point must be, will have 
noticed that this book is in "magazine" layout, organized visually by ideas and 
meanings, for that precise reason. I will be interested to hear whether that has 
worked.) 

And the pity of it is that (like the man in the French play who was surprised to 
learn that he had been "speaking prose all his life and never known it"), we've 
been speaking hypertext all our lives and never known it. 

Now, many writers have tried to break away from sequence. I think of 
Nabokov's Pale Fire, of Tristram Shandy and an odd novel of Julio Cortazar 
called Hopscotch, made up of sections ending with numbers telling you where 
you can branch to. There are many more; and large books generally use many 
tricks to get around the problem of indexing and reviewing what has and hasn'l 
been said or done already. 

However, in my view, a new day is dawning. Computer storage and screen dis- 
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play mean that we no longer have to have things in sequence; totally arbitrary 
structures are possible, and I think that after we've tried them enough people 
will see how desirable they are. 



types of hypertext 

Let's assume that you have a high-power display — and storage displays won't 
do. because you have to see things move in order to understand where they 
come from and what they mean. (Especially text.) So it has to be a refreshed 
CRT. 

Basic or chunk style hypertext offers choices, either as footnote-markers 
(like asterisks) or labels at the end of a chunk. Whatever you point at then 
comes to the screen. 

Collateral hypertext means compound annotations or parallel text. 

Stretchtext changes continuously This requires very unusual techniques, 
but exemplifies how "continuous" hypertext might work. 

Ideally, chunk and continuous and collateral hypertext could all be com- 
bined (and in turn collaterally linked). 

A "fresh" or "specific" hypertext — I don't have a better term at the moment — 
would consist of material especially written for some purpose. An anthological 
hypertext, however, would consist of malcriais brought together from all over, like 
an anthological book. 

A grana hypertext, then, folks, would be a hypertext consisting of "every- 
thing" written about a subject, or vaguely relevant to it, tied together by editors 
(and NOT by "programmers," dammit), in which you may read in all the direc- 
tionsyou wish to pursue. There can be alternative pathways for people who 
think different ways. People who have to have one thing explained to them at a 
time — many have insisted to me that this is normal, although I contend that it is 
a patho-logical condition — may have that; others, learning like true human 
beings, may gather and sift impressions until the ideas become clear. 

And then, of course, you see the real dream. 

The real dream is for "everything" to be in the hypertext. 



Everything you read, you read from the screen (and can always get back to 
right away); everything you write, you write at the screen (and can cross-link te 
whatever you read). 

Paper moulders. Microfilm is inconvenient. In the best libraries it takes ai 
least minutes to get a particular thing. But as to linking them together — footnot- 
ing Aeschylus with Marcus Aurelius, linking genetic data to 15th-century ac 
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A ounts of Indian tribes — well, you can only do it on paper by writing something 
new that ties them together. Isn't that ridiculous? When you could do it all elec- 
tronically in seconds? 

Now that we have all these wonderful devices, it should be the goal of soci- 
ety to put them in the service of truth and learning. And this is the way I propose. 
Not through obscure forms of "information retrieval"; not through newly op- 
pressive forms of "computer-assisted instruction;" and not through a 
purported science of "artificial intelligence" that will create new personalisms to 
irk us. All these obstructive oddities, I think, have developed as separate 
ideais because of the grand preposterosity of Professionalism that has created 
a world-wide cult of mutual incomprehensibility and disconnected special 
goals. Now we need to get everybody together again. We want to go back to 
the roots of our civilization — the ability, which we once had, for everybody who 
could read to be able to read everything. We must once again become a 
community of common access to a shared heritage. 

This was of course what Vannevar Bush said in 1945, in an article 
everybody cites but nobody reads. 

The hypertext solution in many ways obviates some of these other ap- 
proaches, and in addition retains and puts back together the great traditions 
of literature and scholarship, traditions based on the fact that dividing things 
up ar-bitrarily just generally doesn't work. 



the burning bush 

In fact hypertexts were foreseen very clearly in 1945 by Vannevar Bush, Roo- 
sevelt's science advisor. When the war was in the bag, he published a little 
article on various groovy things that had become possible by that time. 

"As We May Think" (Atlantic Monthly, July 1945) is most notable for itf 
clear description of various hypertext techniques — that is, linkages between 
doc uments which may be brought rapidly to the screen according to their 
linkages (So what if he thought they'd be on microfilm.) 

How characteristic of Professionalism. Bush's article has been taken as thi 
starting point for the field of Information Retrieval, but its actual contents havi 



been ignored by acclamation. Information Retrieval folk have mostly done ver 
different things, yet thought they were in the tradition. 

Now people are "rediscovering" the article. If there's another edition of thi 
hnnk T Vione T can run it in entiretv. 
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doug engelbart and "the augmentation of 
intellect" 



Douglas Engelbart is a saintly man at Stanford Research Institute whose dream 
has been to make people smarter and bring them together. His system, on which 
millions of dollars have been spent, is a wonder and a glory. 

He began as an engineer of CRTs; but his driving thought was, quite cor- 
rectly, that these remarkable objects could be used to expand man's mind and im- 
prove each shining hour. 

Doug Engelbart's vision has never been restricted to narrow technical issues. 
From the beginning his concern was not merely to plank people down at display 
consoles, but in the most profound sense to expand man's mind. "The 
Augmentation of Human Intellect," he calis it, by which he means making 
minds work better by giving them better tools to work with. 

An obvious example is writing: before people could write things down, men 
could only learn what they experienced or were told by others in person; writing 
changed all that. Within the computer-screen fraternity, the next step is obvious; 
screens can double and redouble our intellectual capacities. But this is not 
obvious to everybody. Engelbart, patiently instructing those outside, carne up with 
a beautiful example. To show what he meant by the Augmentation of Intellect, 
Engelbart tiedapencil to a brick. Then he actually made someone write with it. The 
result, which was of course dreadful, Engelbart solemnly put into a published re- 
port. Not yet being able to demonstrate the augmentation of intellect, since he had 
as yet no system to show off, he had masterfully demonstrated the dt'saugmenta- 
tion of intellect: what happens if you make man's tools for working out his thoughts 
worse instead of better. As this poor guy was with his brickified pencil, explained 
Engelbart, so are we all among our bothersome, inflexible systems of paper. 

Starting small, Engelbart programmed up a small version of what most fans 
cali "The Engelbart System" some ten years ago. One version has it that when it 
carne to looking for grants, management thought he acted too kooky, and so as- 
signed a Front Man to make the presentation. But, as the story goes, the man from 
ARPA pointed at Engelbart and said, "We want to back him." 



A small but dedicated group at SRI has built up a system from scratch. First 
they used little CDC 1700 minicomputers; then, various grants later, they were 
able to set up their own PDP-10, in which the system now resides, and from 
which it reaches out across the country. 
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Doug calis his system NLS, or "oN-Line System." Basically it is a highly re- 
sponsive, deeply-structured text system, feeding out to display terminals. 
From a terminal you may read anything you or others have written, and write 
with as-yet-unmatched flexibility. 

The display terminals are all over. The project has gone national, though at 
great expense: through the ARPA net of computers, you can in principle be- 
come a user of NLS for something like $50,000 a year. 



mouse? 

The Engelbart Folks have built a pointing device, for telling the system where 
you're pointing on the screen, that is considerably faster and handier than a 
light-pen. (Unfortunately, I don't believe it's commercially available.) It's called 
The Mouse. 

The Engelbart Mouse is a little box with hidden wheels underneath and a 
cable to the terminal. As you roll it, the wheel's turns are signalled to the com- 
puter and the computer moves the cursor on the screen. Ifsfast and 
accurate, and in fact beats a lightpen hands down in working speed. 

Through the command language, NLS allows users to create programs 
that respond in all sorts of ways; thus the fact that certain texthandling styles 
are standard results more from tradition than necessity. 

The same apparently is true of the data structure. I used to be somewhat 
dis-turbed at the way Engelbart's text systems seem to be rigorously 
hierarchical. This in fact is the case, in the sense that having multiple discrete 
leveis is built deep into the system. But it turns out to be harmless. The stored 
text is divided by the storage techniques into multiple leveis, corresponding to 
a Harvard out-line. Think of it as something like this: 

1. HIERARCHICAL FORMAT 

A. STORAGE 

B. DISPLAY 

C. LANGUAGE 

But let's expand this example a little: 



1. HIERARCHICAL FORMAT A. STORAGE 
Al. Everything in NLS is stored with hierarchical codes. A2. Their effect 
depends on the display. 
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B. DISPLAY 

Bl. The hierarchical codes of NLS have no consequences in partit 

ular. B2. The hierarchical codes for NLS can splay the material 
out into; 

variety of display arrangements. B2A. They can be displayed in 
outline form. B2B. They can be displayed in normal text form. B2C. 
These dratted numbers can even be made to disappear. 

C. LANGUAGE 

CI. The command language determines what the display shows a 
the hierarchical structure. C2. What is shown can be determined 

by a program in the commanc 

language. (For instance, "how many leveis down" it is beinj 
shown). 

C2A. This is four leveis down. (The earlier example wasn't.) 
C3. The display format all depends on what display program you 
use, in the NLS command language. 

That's enough of that. I can't help remarking that I still don't like that sort of 
structuring, but it is deep in NLS, and if you don't like it either (poor deprived 
lucky user of NLS) you can program it to disappear, so it's hardly in your way. 

by the beard of the prophet ! 

Engelbart in German means Angelbeard; Doug Engelbart is indeed on the side 
of the angels. In building his mighty system he points a new way for humanity. 
The sooner the better. Any history of the twentieth century will certainly hold him 
high. Few great men are also such nice guys. 



A very basic hypertext system 



Hypertext is non-sequential writing. It's no good to us, though, unless we can go 
instantly in a choice of directions from a given point. 

This of course can only mean on computer display screens. 

Engelbart's system, now, was mainly designed for people who wanted to im- 
merse themselves in it and learn its conventions. Indeed, it might be said to have 
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been designad for a community of people in ciose contact, a sort of system 
of blackboards and collaborative talking papers. 

A more elemental system, with a different slant, was put together at Brown U. 
on IBM equipment. We'll refer to it here as "Carmody's System," after the 
young programmer whose name carne first on the writeup. 

Carmody's system runs on an IBM 360 with 2250 display. While the 2250 is 
a fine piece of equipment, the quirks of the 360's operating system often delay 
the user by making him wait, e.g., for someone else's cards to get punched 
before it re-sponds to bis more immediate uses; this is like making ice-skaters 
wait for oxcarts. 

Anyway, the system essentially imposes no structure on the material; it may 
consist of text segments of any length and ties and links between them. An 
as-terisk appearing anywhere in one piece of text signais a possible jump, 
but the reader doesn't necessarily know where to; sapping the asterisk with 
the lightpen takes you there, however. 



This is stark and simple. It could also get you good and lost. However, a 
simple technique took care of that: every time the user jumped, the address 
of his previous location was saved on a stack. The user also had a RETURN 
button: when he wanted to go back to where he had last jumped from, the 
system would pop the last address off the top of the stack, and take him there. 
(This feature was adapted from my 1967 Stretchtext paper, and turned out to 
work out quite well in practice.) 
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The system also had handy features for light-pen text editing, and variou 
nice printout techniques. All told, it was a clean and powerful design. While i 
lacked higher-level visualization facilities, like Engelbart's display of Leveis or col- 
lateral display, it was in some ways suited for naive users; that is, it was 
eventu-ally fairly safe to use, and could in large part be taught to rank 
beginners in / couple of hours — provided they didn't have to know about JCL 
cards. 

It is left for the reader to figure out interesting uses for it. How would you de 
collateral structures? How could you signal to a reader which of several 
piece! of text ajump was to? 

(At least one real hypertext was actually written on this system. It tied togethei 
a lot of patents for multilayer electrodes. Readers agreed that they could lean 
more from it about multilayer electrodes than they had imagined wanting to 
know.) 
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Personal Dynamic Media" (1977) 
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Dynabook. Gourtesy o/Alan Kay. 

'magine having your own self-contained 
knowledge nanipuiator \n a portable package the 
size and shape ofan irdinary notebook. Suppose it 
had enough power to outrace 'our senses of sight 
and hearing/ enough capacity to store or later 
retrieval thousands of page-equivalents of 
reference vaterials, poems, letters, recipes, 
records, drawings, mmatlons, musical scores, 
waveforms, dynamic umulations, and anyth'mg 
else you would like to remember indchange. " 
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« Alan Kay's goal for the Learning Research Group he founded at Xerox PAR( jn 
the early 1970s was no less than to jnvent the future of communication, anl the 
knowledge tools to make jt possible.This article js condensed from a PAR( 
technical report that documents the six years of research behind the first work 
able hypermedia system. Together with his assistant Adele Goldberg, Kay outlina 
the genesis of Smalltalk, an object-oriented software language that led to the in 
vention of the graphical user jnterface.Their research took root jn the convictiol 
that hypermedia, or "dynamic media," as they cal led jt, represented a profoum 
departure from static media such as painting, television, photography, print pub 
lishing, and film. They saw in hypermedia the radical interactivity that wouk 
characterize Communications in the future. 

The prototype that was the focus of Kay and Goldberg's research, the Dyna 
book, was conceived as a "dynamic medium for creative thought," capable o 
synthesizing all media — pictures, animation, sound, and text — through the jnti 
macy and responsiveness of the personal computer. The Dynabook was, in part 
an interpretation of Vannevar Bush's information storage device, the memex. I 
incorporated the navigational conventions of Douglas Engelbart's oNLine Sys 
tern (the mouse, Windows, and menus in particular). It also borrowed from TeJ 
Nelson's concept of hypermedia. While Kay and his team never built the proto 
type of the Dynabook, by 1974 its key ideas were incorporated into the Xeroj 
Alto computer running the Smalltalk operating system. As a result, the Alto i< 
regarded as the first personal computer, and the original desktop multimedia sys 
tern. 

The Dynabook was meant for everyone, enabling people of all ages to discova 
and express their creativity. It included software such as a text editor, an anj 
mation system, a paint program, and music scoring systems. It was, as the ai| 
thors describe it, a "metamedium," containing within jt all other media. TN 
Dynabook also brought together divergent media through a single interface 
making desktop hypermedia a reality.To prove the accessibility of the device, chil 
dren were recruited to test the prototype machine, and to write original softwar< 
applications such as music and painting programs. Kay's work carne to the at 
tention of young entrepreneurs jn the computer industry, including Steve Jobs o 



Apple Computer and Bill Gates of Microsoft, who profited enormously from tW 
powerful operating system that was designed to give free reign to the intuitivi 
mechanisms of achild's mind. » 
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NTRODUCTION 

"he Learning Research Group at Xerox Palo Alto Research Center is 
concerned dth all aspects of the communication and manipulation of 
knowledge. We de-jgn, build, and use dynamic media which can be used by 
human beings of all ges. Several years ago, we crystallized our dreams into a 
design idea for a per-Dnal dynamic medium the size of a notebook (the 
Dynabook) which could be wned by everyone and could have the power to 
handle virtually all of its owner's iformation-related needs. Towards this goal we 
have designed and built a com-lunications system: the Smalltalk language, 
implemented on small computers we ifer to as "interim Dynabooks." We are 
exploring the use of this system as a pro-ramming and problem solving tool; as 
an interactive memory for the storage and lanipulation of data; as a text editor; 
and as a medium for expression through rawing, painting, animating pictures, 
and composing and generating music. 

We offer this paper as a perspective on our goals and activities during the 
past ears. In it, we explain the Dynabook idea, and describe a variety of 
systems we ave already written in the Smalltalk language in order to give broad 
images of the inds of information-related tools that might represent the kernel 
of a personal omputing medium. 

jackground 

lumans and media. "Devices" which variously store, retrieve, or manipulate in- 
timation in the form of messages embedded in a medium have been in 
existence >r thousands of years. People use them to communicate ideas and 



feelings both ) ouiers and back to themselves. Although thinking goes on in 
one's head, ex-:rnal media serve to materialize thoughts and, through 
feedback, to augment the :tual paths the thinking follows. Methods 
discovered in one medium provide letaphors which contribute new ways to 
think about notions in other media. 

For most of recorded history, the interactions of humans with their media 
ave been primarily nonconversational and passive in the sense that marks 
on aper, paint on walls, even "motion" pictures and television, do not change in 
re-x>nse to the viewer's wishes. A mathematical formulation — which may 
sym-olize the essence of an entire universe — once put down on paper, 
remains static id requires the reader to expand its possibilities. 

Every message is, in one sense or another, a simulation of some idea. It 

may 
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be representational or abstract. The essence of a medium is very much 
dependen! on the way messages are embedded, changed, and viewed. 
Although digital com-puters were originally designed to do arithmetic 
computation, the ability to simulate the details of any descriptive model means 
that the computer, viewed as a medium itself, can be all other media if the 
embedding and viewing methods are sufficiendy well provided. Moreover, this 
new "metamedium" is active — it can re-spond to queries and experiments — so 
that the messages may involve the learnei in a two-way conversation. This 
property has never been available before excepl through the medium of an 
individual teacher. We think the implications are vasl and compelling. 

A dynamic medium for creative thought: the Dynabook. Imagine having 
your own self-contained knowledge manipulator in a portable package the 
size and shape of an ordinary notebook. Suppose it had enough power to 
outrace your senses of sight and hearing, enough capacity to store for later 
retrieval thou-sands of page-equivalents of reference materials, poems, letters, 
recipes, records, drawings, animations, musical scores, waveforms, dynamic 
simulations, and any-thing else you would like to remember and change. 

We envision a device as small and portable as possible which could both 
take in and give out information in quantities approaching that of human 
sensory systems. Visual output should be, at the least, of higher quality than 
what can be obtained from newsprint. Audio output should adhere to similar 
high-fidelit)i standards. 

There should be no discernible pause between cause and effect. One of 
the metaphors we used when designing such a system was that of a musical 
instru-ment, such as a ilute, which is owned by its user and responds instandy 
and con-sistently to its owner's wishes. Imagine the absurdity of a one- 
second dela) between blowing a note and hearing it! 



These "civilized" desires for flexibility, resolution, and response lead to the 
conclusion that a user of a dynamic personal medium needs several 
hundred times as much power as the average adult now typically enjoys from 
timeshared computing. This means that we should either build a new resource 
several hundred times the capacity of current machines and share it (very 
difficult and ex-pensive), or we should investigate the possibility of giving each 
person his owi powerful machine. We chose the second approach. 
Design background. The first attempt at designing this metamedium (die 
FLE> machine) occurred in 1967-69. Much of the hardware and software 
was sue cessiul from the standpoint of computer science state-of-the-art 
research, buj 
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Iked sufficient expressive power to be useful to an ordinary user. At that time 
e became interested in focusing on children as our "user community." We were 
reatly encouraged by the Bolt Beranek and Newman/MIT Logo work that uses 
i robot turtle that draws on paper, a CRT version of the turtle, and a single 
music 

;enerator to get kids to program. 

Considering children as the users radiales a compelling excitement when 
iewed from a number of different perspectives. First, the children really can 
vrite programs that do serious things. Their programs use symbols to stand 
for ibjects, contain loops and recursions, require a fair amount of visualization of 
al-ernative strategies before a tactic is chosen, and involve interactive 
discovery ind removal of "bugs" in their ideas. 

Second, the kids love it! The interactive nature of the dialogue, the fact that 
hey are in control, the feeling that they are doing real things rather than 
playing mh toys or working out "assigned" problems, the pictorial and auditory 
nature >f their results, all contribute to a tremendous sense of accomplishment 
to their xperience. Their attention spans are measured in hours rather than 
minutes. 

Another interesting nugget was that children really needed as much or more 
omputing power than adults were willing to settle for when using a timesharing 
vstem. The best that timesharing has to offer is slow control of crude wire-frame 
jeen-tinted graphics and square-wave musical tones. The kids, on the other 
tand, are used to finger-paints, water colors, color television, real musical in- 
truments, and records. If the "medium is the message," then the message of 
3w-bandwidth timesharing is "blah." 



V.\ interim dynabook 



\e have designed an interim version of the Dynabook on which several inter- 
sting systems have been written in a new medium for communication, the 
mahulk\jTO< A m\am A lm< AA 

vith more than 200 users, most notably setting up a learning resource center 
in 

i local junior high school. 

The interim Dynabook is a completely self-contained system. To the user, it 
ippears as a small box in which a disk memory can be inserted; each disk 
con-ains about 1500 page-equivalents of manipulable storage. The box is 
connected o a very crisp high-resolution black and white CRT or a lower- 
resolution high-[uality color display. Other input devices include a 
typewriter keyboard, a chord" keyboard, a pointing device called a "mouse" 
which inputs position as 
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it is moved about on the table, and a variety of organ-like keyboards for playiil 
music. New input devices such as these may be easily attached, usually withoi 
building a hardware interface for them. Visual output is through the display, ai 
ditory output is obtained from a built-in digital-to-analog converter connected I a 
standard hi-fi amplifier and speakers. 

We will attempt to show some of the kinds of things that can be done with 
Dynabook; a number of systems developed by various users will be briefly illu 
trated. 

Remembering, seeing and hearing. The Dynabook can be used as an 
intera tive memory or file cabinet. The owner's context can be entered 
through a ke board and active editor, retained and modified indefinitely, and 
displayed o demand in a font of publishing quality. 

Drawing and painting can also be done using a pointing device and an icon 
editor which allows easy modification of pictures. A picture is thus a manipul 
ble object and can be animated dynamically by the Dynabook's owner. 

A book can be read through the Dynabook. It need not be treated as a sina 
lated paper book since this is a new medium with new properties. A dynam 
search may be made for a particular context. The non-sequential nature of the fi 
medium and the use of dynamic manipulation allows a story to have many a 
cessible points of view; Durrell's Alexandria Quartet, for instance, could be of 
book in which the reader may pursue many paths through the narrative. 



Different fonts for different effects. One of the goals of the Dynabook's desij 
is not to be worse than paper in any important way Computer displays of the pa 
have been superior in matters of dynamic writing and erasure, but have failed 
contrast, resolution, or case of viewing. There is more to the problem than ju 
the display of text in a high-quality font. Different fonts create different moa 
and cast an aura that influences the subjective style of both writing and readin 
The Dynabook is supplied with a number of fonts which are contained on the £ 
storage. 

The Dynabook as a personal medium is flexible to the point of allowing j 
owner to choose his own ways to view information. Any character font can be d 
scribed as a matrix of black and white dots. The owner can draw in a charad 
font of his own choosing. He can then immediately view font changes within ti 
context of text displayed in a window. With the Dynabook's fine grain of dispt 
the rough edges disappear at normal viewing distance to produce high-quali 
characters. 
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iditing. Every description or object in the Dynabook can be displayed and 
idited. Text, both sequential and structured, can easily be manipulated by com- 
jining pointing and a simple "menu" for commands, thus allowing deletion, 
ransposition, and structuring. Multiple windows allow a document (composed 
>f text, pictures, musical notation) to be created and viewed simultaneously at 
jeveral leveis of refinement. Editing operations on other viewable objects 
(such is pictures and fonts) are handled in analogous ways. 

Filing. The multiple-window display capability of Smalltalk has inspired the 
lotion of a dynamic document. A document is a collection of objects that have 
a jensory display and have something to do with each other; it is a way to store 
and »etrieve related information. Each subpart of the document, or frame, 
has its >wn editor which is automatically invoked when pointed at by the 
"mouse." Fhese frames may be related sequentially, as with ordinary paper 
usage, or inserted with respect to properties, as in cross-indexed file 
systems. Sets which an automatically map their contents to secondary 
storage with the ability to brm unions, negations, and intersections are part of 
this system, as is a "mode-ess" text editor with automatic right justification. 

The current version of the system is able to automatically cross-file several 
housand multifield records (with formais chosen by the user), which include or- 



linary lexlual documenls indexed by conteni, the Smalltalk system, personal 
jles. diagrams, and so on. 

)rawing/painting. The many small dots required to display high-qualily char- 
cters (about 500,000 for an 8-1/2" X 11" sized display) also allow skelching- 
uality drawing, "halftone painting," and animation. The subjective effecl of gray 
cale is caused by the eye fusing an area containing a mixture of small black 
and .'hite dots. A brush can be grabbed with ihe "mouse," dipped into a paint 
pol, nd then the halftone can be swabbed on as a funclion of ihe size, shape, 
and ve->city of the brush. 

Curves are drawn by a pen on the display screen. (Straight lines are 
curves iih zero curvature.) In the Dynabook, pens are members of a class 
that can se-rctively draw with black or white (or colored) ink and change ihe 
ihickness of le irace. Each pen lives in its own window, careful not to traverse 
its window oundaries but to adjust as its window changes size and position. 

Jiimation and music. Animation, music, and programming can be thoughl 
of ; different sensory views of dynamic processes. The structural similarities 
among 
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them are apparent in Smalltalk, which provides a common framework for es 
pressing those ideas. 

All of the systems are equally controllable by hand or by program. Thiu 
drawing and painting can be done using a pointing device or in conjunclia 
with programs which draw curves, fill in areas with tone, show perspectives ( 
three-dimensional models, and so on. Any graphic expression can be 
animatec either by reflecting a simulation or by example (giving an "animator" 
program sample trace or a route to follow). 

Music is controlled in a conipletely analogous manner. The Dynabook can a< 
as a "super synthesizer" getting direction either from a keyboard or from 
"score." The keystrokes can be captured, edited, and played back. Timbres, 
di "fonts" of musical expression, contain the quality and mood which different 
ir struments bring to an orchestration. They may be captured, edited, and 
used namically. 

simulation 

In a very real sense, simulation is the central notion of the Dynabook. Each of ti 
previous examples has shown a simulation of visual or auditory media. Here 
ai a number of examples of interesting simulations done by a variety of 
users. 

An animation system programmed by animators. Several professional 
anima tors wanted to be able to draw and paint pictures which could then be 
animal» in real time by simply showing the system roughly what was wanted. 
Desirei changes would be made by iconically editing the animation 
sequences. 

Much of the design of SHAZAM, their animation tool, is an automation of th 
media with which animators are familiar: movies consisting of sequences c 
frames which are a composition of transparent eels containing foreground ani 
background drawings. Besides retaining these basic concepts of convention: 
animation, SHAZAM incorporales some creative supplementary capabilities. 
Animators know thai ihe main aclion of animation is due not to an individ ual 
frame, but to the change from one frame to the next. It is therefore much eai ier 
lo plan an animalion if it can be seen moving as it is being crealed. SHAZAI 
allows any eel of any frame in an animation to be edited while ihe animalion is i 
progress. A library of already-crealed eels is maintained. The animation can b 



single-stepped; individual eels can be reposilioned, reframed, and redrawn; nei 
frames can be insertad; and a frame sequence can be created at any time by 
al 
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ling the eel to the pointing device, then showing the system what kind of 
vement is desired. The eels can be stacked for background parallax; hales 
d windows are made with transparent paint. Animation objects can be painted 
programs as well as by hand. The control of the animation can also be easily 

)ne from a Smalltalk simulation. For example, an animation of objects bounc- 
ig in a room is most easily accomplished by a few lines of Smalltalk that express 
ic class of bouncing objects in physical terms. 

L drawing and painting system programmed by a child. One young girl, who 
iad never programmed before, decided that a pointing device ought to let her 
Iraw on the screen. She then built a sketching tool without ever seeing ours. 
She onstantly embellished it with new features including a menu for brushes 
se-2Cted by pointing. She later wrote a program for building tangram designs. 

This girl has taught her own Smalltalk class; her students were seventh- 
raders from her junior high school. One of them designed an even more elabo- 
ate system in which pictures are constructed out of geometric shapes created 
by ointing to a menu of commands for creating regular polygons. The polygons 
can nen be relocated, scaled, and copied; their color and line width can 
change. 

ai audio animation system programmed by musicians. Animation can be 
onsidered to be the coordinated parallel control through time of images con- 
eived by an animator. Likewise, a system for representing and controlling mu- 
ical images can be imagined which has very strong analogies to the visual 
world, lusic is the design and control of images (pitch and duration changes) 
which can e painted diherent colors (timbre choices)-, it has synchronization 
and coordi-ation, and a very ciose relationship between audio and spatial 
visualization. 

The Smalltalk model created by several musicians, called TWANG, has the 
otion of a chorus which contains the main control directions for an overall piece. 
I chorus is a kind of rug with a warp of parallel sequences of "pitch, duration, 
nd articulation" commands, and a woof of synchronizations and global 
direc-.ves. The control and the player are separate: in SHAZAM, a given 
movie sequence can animate many drawings; in TWANG, a given chorus 
can tell many lifferent kinds of instrumentalists what should be played. These 
voices can be ynthetic timbres or timbres captured from real instruments. 
Musical effects such s vibrato, portamento, and diminuation are also 
available. 



A chorus can be drawn using the pointing device, or it can be captured by 
ilaying it on a keyboard. It can be played back in real time and dynamically 
dited in a manner very similar to the animation system. 
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We use two methods for real-time production of high-quality timbres; bo 
allow arbitrary transients and many independent parallel voices, and are coi 
pletely produced by programs. One of these allows independent dynamic co 
trol of the spectrum, the frequency, the amplitude, and the particular collecti< 
of partiais which will be heard. 

For children, this facility has a number of benefits: the strong similarities I 
tween the audio and visual worlds are emphasized because a single vernacul 
which actually works in both worlds is used for description; and second, the ai 
and skills of composing can be learned at the same time since tunes may 
drawn in by hand and played by the system. A line of music may be copit 
stretched, and shifted in time and pitch; individual notes may be edited. Imi 
tive counterpoint is thus easily created by the fledgling composer. 

A musical score capture system programmed by a musician. OPUS is a 
u 

sical score capture system that produces a display of a conventional musii 
score from data obtained by playing a musical keyboard. OPUS is designed 
allow incremental input of an arbitrarily complicated score (full orchestra wi 
chorus, for example), editing pages of the score, and hard copy of the final res 
with separate parts for individual instruments. 



conclusion 

What would happen in a world in which everyone had a Dynabook? If such a n 
chine were designed in a way that any owner could mold and channel its pow 
to his own needs, then a new kmd of medium would have been created 
metamedium, whose conteni would be a wide range of already-existing and rt 
yet-invented media. 

An architect might wish to simulate three-dimensional space in order to pen 
and edit his current designs, which could be conveniently stored and ero 
referenced. 

A doctor could have on file all of his patients, his business records, a di 
reaction system, and so on, all of which could travel with him wherever went. 



A composer could hear his composition while it was in progress, particula if 
it were more complex than he was able to play. He could also bypass the 
credibly tedious chore of redoing the score and producing the parts by hand 
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Learning to play music could be aided by being able to capture and hear 
one's own attempts and compare them against expert renditions. The ability to 
express music in visual terms which could be filed and played means that the 
acts of composition and self-evaluation could be learned without having to wait 
for tech-nical skill in playing. 

Home records, accounts, budgets, recipes, reminders, and so forth, could 
be easily captured and manipulated. 

Those in business could have an active briefcase which travelled with them, 
containing a working simulation of their company, the last several weeks of cor- 
respondence in a structured cross-indexed form — a way to instantly calculate 
profiles for their fiitures and help make decisions. 

For educators, the Dynabook could be a new world limited only by their 
imagination and ingenuity. They could use it to show complex historical inter- 
relationships in ways not possible with static linear books. Mathematics could 
be-come a living language in which children could cause exciting things to 
happen. Laboratory experiments and simulations too expensive or difficult to 
prepare could easily be demonstrated. The production of stylish prose and 
poetry could be greatly aided by being able to easily edit and file orie's own 
compositions. 

These arejust a few ways in which we envision using a Dynabook. But if the pro- 
jected audience is to be "everyone," is it possible to make the Dynabook 
gener-illy useful, or will it collapse under the weight of trying to be too many 
different tools for too many people? The total range of possible users is so 
great that any ittempt to specifically anticipate their needs in the design of the 
Dynabook would znd in a disastrous feature-laden hodgepodge which would 
not be really suitable Por anyone. 

Some mass Items, such as cars and television sets, attempt to anticipate 
and provide for a variety of applications in a fairly inflexible way; those who 
wish to do something different will have to put in considerable effort. Other 
Items, such is paper and clay, offer many dimensions of possibility and high 
resolution; these :an be used in an unanticipated way by many, though tools 
need to be made or Dbtained to stir some of the medium's possibilities while 
constraining others. 



We would like the Dynabook to have the flexibility and generality of this second 
kind of Item, combined with tools which have the power of the first kind. Thus 
i great deal of effort has been put into providing both endless possibilities 
and :asy tool-making through the Smalltalk programming language. 
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Our design strategy then, divides the problem. The burden of system desigi 
and specification is transferred to the user. This approach will only work if we di a 
very careful and comprehensive Job of providing a general medium of cora 
munication which will allow ordinary users to casually and easily describe thei 
desires for a specific tool. We must also provide enough already-written genera 
tools so that a user need not start from scratch for most things she or he may wisl 
to do. 

We have stated several specific goals. In summary, they are: 

3. to provide coherent, powerful examples of the use of the Dynabook in an 
across subject areas; 

4. to study how the Dynabook can be used to help expand a person's visuj 
and auditory skills; 

5. to provide exceptional freedom of access so kids can spend a lot of tini 
probing for details, searching for a personal key to understanding processithey 
use daily; and 

6. to study the unanticipated use of the Dynabook and Smalltalk by childrein 
all age groups. 



Richard A. Bolt 
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Spatial Data-Management. Courtesy 
oTHans-Christian Liscnewski. 



" We have attempted to create an interface which 
is not a tiny, narrow-band A port-hole'into an 
Information bank, that bank jtselfan abstractly 
addressedset of intangibles. Rather, we have 
attempted radically to recast the setting as an 
y mformational surround' wherein the user js 
directly engaged with data bodied forth in vision, 
sound, and touch, data inhabiting a spatially 
definite Virtual' world that can be interactively 
explored and navigated. " 
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« Richard Bolt is one of the most influential figures jn the field of multimodj 
interfaces — interfaces that enable the user to interact with a computer throug 
manual gesture and speech, rather than a keyboard and mouse. In the 1970s h 
was a member of the Architecture Machine Group at MIT, the precursor to til 
Media Lab and a center for many of the technological breakthroughs and cor 
ceptual paradigms that define human-computer interface. At Arch-M AC, as tij 
center was called, Bolt designed the legendary "media room" under the directia 
ofhis ciose collaborator Nicholas Negroponte.The "media room" placed the us( in 
an environment that was itself a computer interface that reacted to voice, ge ture, 
and gaze.The user sat in a chair, pointed at objects that appeared on a wa| sized 
screen, and spoke commands that moved computer-generated objects 1 specified 
locations. They called jt the "Put-That-There" system. Bolt continue this research 
as a founding member of the Media Lab, heading its Advance Human Interface 
Group. 

While heralded for "Put-That-There," Bolt and Negroponte are less knovn for 
the work represented in this groundbreaking paper from the same pericn which 
describes the "media roomV'spatial data-management system, or SDM5 This was 
the first hypermedia system to organize information for retrieval withi a 3-D 
virtual environment — to offer, in Bolt's words, "a multidimensional wii dow into 
data." SDMS took a significantly different approach to hypermedi than the one 
pursued by Vannevar Bush,Ted Nelson, and Douglas Engelbart, wh had conceived 
of it as a way to link media objects to one another through a Index. Alan Kay 
continued in this tradition when he introduced the GUI, whj< used icons 
arranged on a fiat "desktop" surface to represent bodies of inform, tion. SDMS, 
however, drew on developments jn virtual reality, which led Bolt ar Negroponte to 
consider how data might be presented as symbols within an q gaging, 
multisensory, navigable environment. 

The premise was simple. As Bolt writes, the "well-evolved human ability 1 
organize information spatially remains essentially untapped in the realm < 
computer-based information handling." SDMS relied on what Bolt called "til 
Simonides Effect." Simonides, the ancient Greek poet famous for his memor 
would divide his poems into parts, and learn each at a different location in a ten 
ple.To recall the text, he would visualize himself walking through the temple an 
the parts would come back to him in sequence. Bolt and Negroponte used th 
"ancient principle of using spatial cueing as an aid to performance and memory as 
a model for the information landscape they developed for SDMS, which trw 



referred to as"Dataland." Significantly, the SDMS allowed usersto make not« of 
their observations as they pursued their own trails of association through th 
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system The Dataland was not only navigable, it could be interacted with. A half 
decade before William Gibson popularized theconcept of "cyberspace/' Boltand 
Negroponte had built a working prototype that gave the user the freedom to 
explore and engage with an architecturally conceived hypermedia 
environment.)) 

the concept 

The basis of spatial data-management is accessing a data Item by going to 
where it is rather than referencing it by name. 

Consider what occurs when you retrieve a book in your home library. You 
look at the left-hand bookcase, second shelf from the top, say, and sean about 
two-thirds down the shelf for a certain book spine, Thus, the book is located 
not by its title but by its customary site in a space defined by other books and 
the bookcase itself. 

Consider how people ordinarily find Items on their desktops: the appoint- 
ment book is up and to the right; the telephone is in the lower right corner; 
high priority memos are kept in an "in" box immediately to the left of the desk 
blot-ter; less urgent Items are in the file folder, top middle of desk; and so 
forth. 

The person who uses this desk has organized the layout of Items in a 
more or less systematic way. He or she refers to them constantly throughout 
the working day: reaching in that direction, that far, up, down, to the right, to 
the left. Through this activiry, a mental image of the layout of the desk is 
elaborated in the "mind's eye." Additionally, through constant tactile interaction 
with the Items, reaching for and touching them, a "motor memory" of where 
things are arises as well. A script for the act of retrieval becomes encoded into 
the musculature, as it were, according to where the Item is located. 

The practical importance of this spatial principle of organization is illus- 
trated when someone happens to disturb our familiar arrangement of 
things. Perhaps some well-meaning soul tries to tidy up our seemingly chaotic 
desktop. Rather than being a benefit, the now "well-organized" desk is for us 
an organi-zational disaster: the former "messy" arrangement was a familiar 
and well trav-eled ground whereupon we could go directly to what we wanted. 
Such a loss of a familiar spatial layout can impress upon us the extent to which 



we take this sub-tie but powerful principle of organization for granted in our 
daily lives. 

An even more striking instance of the role of spatiality in common experience 
is the coherence and persistence of "imagery." Two individuals, arguing a 
topic in front of a blackboard, will refer each other to diagrams, equations, 
and terms on the basis of where they had been written, even long after they 
have been erased! 
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It is surprising how pervasive the underlying notion of spatiality is, even i] 
symbolic modes of thought. Consider a file of three-by-five notecards, organiza 
alphabetically, but without letter tabs showing. "Filed under the letter R" traia 
lates into a tactile estimate of how far down the row of cards we must reach. Th 
same holds true for a dictionary without thumb indexes. When we want to loo! 
up a word beginning with the letter .h, we "guesstimate" where we should opa 
the pages of the book to begin our search. Thus, R is somehow a relative 
distanc* as well as a letter. There is more spatiality implicit in what is ordinarily 
thougk oias symbolic retrieval than we may realize. 

In summary, it is precisely the insight about how we tend to retrieve itera 
from desktops, from files and bookshelves, even from erased blackboards, that 
lie at the heart of the spatial data management concept: we find Items on the 
basi of a more or less defmite sense of their location in a familiar space, which 
spac may be actually present or remembered. 

This well-evolved human ability to organize information spatially remains a 
sentially untapped in the realm of computer-based information handling. Typ 
cally, in such systems, retrieval on a symbolic or name basis is the norm, and 
muj be the norm because the conventional keyboard interface is too limited a 
chai nel, the wrong mode and medium, to begin to offer the user a direct, 
palpabl sense of spatiality. 

In the spring of 1976, the Architecture Machine Group at MIT proposed I 
the Cybernetics Technology Office of the United States Defense Advanced Ri 
search Projects Agency (DARPA) a program of research organized under ti 
title "Augmentation of Human Resources in Command and Control throug 
Multiple Media Man-Machine Interaction." Intrinsic to the ensemble of studu 
outlined in the proposal was a study recalling the ancient principle of using spj 
tial cueing as an aid to performance and memory: the "Simonides Effect." 

Simonides was a poet of ancient Greece famous for bis ability to give Ion 
recitations entirely from memory. His secret, which as a teacher of rhetoric 1 
shared with his students, was to tie each successive part of a to-be- 
remembere poem or speech to a specific locale within the mental floor plan of 



either an a tual or imagined temple. Around the floor of the temple were 
statues, serving I augment the imagery of specific niches or corners. 

To commit to memory a long poem or speech, the orator would pause in h 
imagination before the threshold, and with this mental image in mind, commit ti 
introductory remarks to memory. Then, for each successive subsection of the ta 
to be given, the orator would mentally walk from place to place within the tei 
pie, rehearsing the appropriate material before some specific piece of statuai 



"Spatial Oata-Management" 189 

The resultant path about the temple would then serve as a mental schema to or- 
ganize the speech upon its eventual recitation, the speech being retrieved and re- 
constructed during oration by way of an imaginary tour from statue to statue 
around the temple floor. 

Another study outlined in that same proposal looked to the exploitation of virtual 
spatiauty as a matrix for organizing information: the "virtual bulletin board." This 
term refers to the formulation of an implicit space which occupies no "real" space 
but can nonetheless contain a great deal, as well as offer the same spatial or- 
ganizational cueing as to the whereabouts of material that real work surfaces do. 
The tacit size of a "field" of information to be portrayed in computer graphics need 
not be limited by the size of the physical display. Since it is virtual, its apparent size 
and graphic organization are open to user definition in the unconstrained sense 
of Ivan Sutherland's characterization of a computer display as "a window on 
Alice's Wonderland" ("Computer Displays" Scientific American, June 1970). 

These two spatial themes specifically resonated with Dr. Craig Fields, Senior 
Program Manager in the Cybernetics Technology Office. As a result we started to 
explore the potentialities of a spatially oriented data management system, "a mul- 
tidimensional window into data." The outcome has been that over the past two 
years, under the aegis of DARPA, the Architecture Machine Group has been ex- 
perimenting with the creation of an information-management system whose dis- 
tinguishing characteristic is that it exploits the user's sense of spatiality for purposes 
of organizing and retrieving data: a spatial data-management system, orSDMS. 



the setting 

The setting of our experimental version of SDMS is a multiple media information 
place. It can be construed as an image of an office of the future. 



The physical aspects of our "media room" are easily described. The media 
room is about the size of a personal office. The focal point of the room is a wall- 
sized display screen. Directly in front of this large screen is an instrumented 
Eames chair. On either arm of the chair is a small, pressure-sensitive "joystick." 
A.djacent to the joystick on each arm is a two-inch-square touch-sensitive pad. 
\lso associated with the chair as complementary instrumentation is a ten-inch- 
iquare data tablet, with stylus; the tablet is upholstered to serve as a lap pad. This 
ablet bears added instrumentation: a small microphone mounted at its top edge 
o allow voice input. 

Two small television monitors are situated one on either side of the chair. 
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Each of these monitors is touch-sensitive so that the user can point to data or 
can input gestures indicating actions to be taken. 

Finally, eight loudspeakers provide sound output for the media room. Four 
of them are placed around the perimeter of the large-format display to the 
users front. The remaining four are correspondingly arrayed to the user's 
rear. Octa-phonic sounds can be presented to the user as coming from 
various apparent di-rections in the space of the room, in coordination with 
actions on the visual display. The rate of playback is variable so that dynamic 
sound/motion effects, such as Doppler shifts, may be achieved. We cali these 
"sound-synch graphics." 

Notice that there is a conspicuous, but not mandatory, absence of a 

keyboard. 

The configuration described is one which evolved through previous work for 
the Office of Naval Research (Contract number 0014-75-C-0460) and over 
the course of our experimentation, but this precise ensemble is certainly not the 
only one that might have been adopted. The kernel notion of managing data 
spatially is not necessarily tied to a room-sized terminal into which a user 
goes versus a desktop arrangement in front of which he or she sits. 

The relaxed ambiance conveyed by the presence of a chair of the Eames 
genre in lieu of something more utilitarian is intentional, however. It reflects con- 
victions and positions about the nature and tone of human-computer 
interaction that we have attempted to actualize in the media room setting. Just 
as the hands-on immediacy of touch-sensitive pads suggests a literal 
impatience with intangibles about data, so the decor as epitomized in the 
selection of the style of chair rebuts the premise that system users must live 
in severe, ascetic settings. 

We have attempted to create an interface which is not a tiny, narrow-band 
"port-hole" into an information bank, that bank itself an abstractly addressed 
set of intangibles. Rather, we have attempted radically to recast the setting as 



an "in-formational surround" wherein the user is directly engaged with data 
bodied forth in vision, sound, and touch, data inhabiting a spatially definite 
"virtual" world that can be interactively explorad and navigated. 



finding data 

Spatial data-management involves the creation of a plausible and 
commodious "virtual" space at the computer interface, together with a way of 
getting around in that space quickly and easily. 

The spatial world of SDMS consists of a single plane. This surface, called 
"Dataland," is presented to the user in two aspects: (1) it is continuously visible 
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to the user in its entirety in an "aerial," top-on view displayed on one of the 
mon-itors, arbitrarily situated to the user's right, called the "world view" 
monitor; and (2) simultaneously, a small subsector of the Dataland surface is 
displayed on the ten-foot diagonal screen to the user's front, vastly enlarged 
and with appre-ciable gain in detail. 

The logical relationship between these two views is akin to a mapping key, 
which places a particular image within a larger locale. The large screen 
effectively functions as a "window" and, as we shall see later, a "magnifying 
glass" onto Dataland. 

The "world view" monitor serves specifically as a navigational aid to the user 
in getting around Dataland. The large display of whatever portion of Dataland 
is so "ciose up" that the user would get lost easily if there were not always on 
view a map of the entire Dataland world. A small, highlighted, "you-are-here" 
rectan-gle on that display shows the user at all times the position of the large- 
screen window on Dataland. 

To illustrate how an actual Dataland might appear to a user seated in the 
media room's control chair, let us assume a database already created. In 
practice, individuals would design and structure their own Datalands 
according to personal preferences, tasks, and needs, just as the contents and 
organization of any-one's desktop come to reflect some combination of Job 
requirements and individual taste. 

Now, what is visible to the user on the surface of this example of Dataland? 

items of various sorts may be seen. Soine are small photographs: pictures of 
animais, of people, even a miniature LANDSAT satellite photo of New 
England. Other Items look like small maps. We see what appear to be book 



covers, television sets, a business letter, even a small image of an electronic 
hand-held calculator. Yet other items resemble emblems or "glyphs," not 
dissimilar from business logos. 

How does one get around in such an informational landscape? 

User control of navigation around Dataland is provided by "joystick" action. 
The small joystick in the right arm of the chair permits a helicopter-like flight over 
the surface. The joystick itself does not deflect, but responda to pressure. It 
is "self-centering" in that it registers zero deflection when not touched. Push 
the stick "away" from you and the helicoptering motion over Dataland is north, 
or toward the top of the screen. Pull the joystick toward you and motion is 
south. Pressure to the right and left will direct you east or west. 

The apparent rate of travel is proportional to the pressure the user exerts 
on the joystick. At maximum pressure, it takes about twenty seconds to travel 
across Dataland. . 
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Perusing data 

Logically distinct {romfinding a certain item in Dataland is perusing that item in 
place. Once the user has found the item and zoomed in on it, the specific 
forra that perusal takes is conditioned by the nature of the data. 

In direct contrast to material that is "out there," visible, to be developed in full 
simply by approach, are malcriais which need to be "perused," where perusal 
means opening up a volume, starting a film clip, selecting part of a set of 
sudes to be seen, and so on. The essential idea is that the user opens up and 
develops his interchange with the data more or less interactively, through 
direct touch in a "hands-on" manner. The dictionary sense of "perusal" is that 
of "survey or scrutiny, in some thoroughness and detail." It is this engaged 
quality of person-data interchange that we intend by this terminology. 

Our notion of perusal includes a class of "data types" which, when 
addressed, prove to be unusually rich in motion, color, and sound. One of 
these new data types is yet an oid and familiar one: the "book." 

The user zooms in on what appears from afar to be simply a rectangle. At 
ciose enough range, the color patch gains detail in the form of some 
horizontal black rows of "characters" implying a printed title. Upon even yet 
nearer approach, the black figures give way to readable text, giving the title of 
the item, which is now clearly seen as implying a "book-like" image. 

The user, of course, knows that the rectangle is a "book," having put it there 
in the first place. However, if he is a subscriber to a computer-network-based 
"book of the month" service, and has perhaps requested or has a standing 



ordei for books, the book may be one he has yet to see, having just 
appeared on the Dataland surface in a certain area on the data plane serving 
as a "book drop" foi incoming Communications which are book-like. 

Upon nearest approach, when the image of the book is fully addressed in 
thl visual sense of filling the large screen, a "table of contents" for the book 
appean on the second and so far unused color monitor to the user's left. This 
table-of contents view is one of a class of interactive "key maps" made available 
to the use as adjuncts to certain data types, to permit direct user interaction 
with the dac to be perused. 

In this instance, the key map allows user access to the interior of the "book' on 
display, exactly as the table of contents of a "real" book lets the reader (a) knov 
what the book offers by way of topics covered, and (b) serves as a 
"dispatching 1 
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table to send the reader direcdy to the subsection of the book that is of most 
im-mediate interest. 

The table of contents can "expand" to show a finer breakdown of conteni 
area when the user selecls a major heading of interest. In concert with the 
activity on the key map monitor screen, the main large-screen view will "go to" 
the section of the book selected by the user through the interactive key map 
so that the user may read the material. 

There is yet further interaction with the data type of the book: that of turn- 
ing its pages. This action is initiated by a specific user action: a page- 
lurningges-fure given as right-to-left, top-to-bottom stroke on either of the 
touch-sensitive pads on the arms of the user chair. Any page can be turned 
back by a stroke across the pad in the opposite direction. The accompanying 
visual action is the display of an "animation" of a page actually turning. The 
upper right-hand corner of the page on view progressively "lifts" away and 
sweeps leftwards across the screen immediately revealing the new page 
below. 

The intent is not to give a nostalgic impression of the way books used to 
look, like the electric fireplace with plastic back-lit logs. The purpose is quite 
functional. First, the page is a unit of apprehension of the book; it is a "bite- 
sized" chunk or packet of the material that one feels like one can take in 
com-fortably, in contrast to the stretching-ever-onward, rhythmless 
character of scrolled text. Second, the page serves as a progress marker 
through the text in a "milepost" sense. With continuous scrolling, such 



natural, discrete units of progress are not explicit in the medium, and, if felt 
necessary by the reader, must be efibrtfuHy abstracted, resulting in a more 
burdensome tone to the reading task. 

Thus, the "page-flipper" is not frivolous. It provides the user with feedback 
that a page has turned, the directionality of the animation corresponding to 
for-ward or backward. Specifically, it reinstales for the user, in the context of 
the reading of computer- television texl, ihe functional equivalen! of what 
Uiefact of the page gives to the reader of traditional paged printed media: the 
sense of knowing where you are in the material. More generally, it is an 
instance of whal we choose lo term a "media fiducial," a marking system, or 
buill-in fealure of any medium ihal serves ihe funclion of lelting you know 
"where you are" based on locational cues arising out of ihe basic structure of 
ihe medium itself. 

One radical departure from the ordinary concept of the book, except as 
ad-umbraled in such page-corner flip-animalions as ihis reporl has in ils upper 
lefl-hand page corners, is the fact ihal while some page uluslralions will be still 
sudes, olher uluslralions will be "movies," or animaled diagrams, with or without 
sound 
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accompaniments. The possibility of sound can, upon occasion, pertnit the bo< 
to comment upon its own text and illustrations, and even read itself to you. 

Another unusual data type residing in Dataland is that of television. An ii 
stance of this is the presence of a "glyph" on the Dataland surface which is ti 
image of a Sony television. Zoomed in on, the television set becomes very larg 
its own screen filling the large screen directly before the user. What is at first 
rather sketchy black-and-white image on the tube face of this "virtual television 
is then replaced with live action: Columba, The Sting, or a documentary on ha 
to stop smoking. 

What may be seen on this television set in Dataland is anything that can I 
put on television, live or recorded, closed-circuit or broadcast. The user couk 
for example, indicate in the context of a session with SDMS that he next wan 
to view the twelve o'clock news, or see a rerun of some footage that was taped 
ea I ier 

"Perusing" implies the scrutiny in detail of items. This scrutiny in detail a an 
interactive process surely will trigger in the user the impulse to annotate, to a\ 
pendjottings, to comment "marginally." The traditional properties of compuU 
media typically frustrate this impulse, one so readily indulged in on papen 

As an antidote to this shortcoming, we have introduced the capacity for mal 
ing written annotations in "transparent ink," of varying line widths and colon 



The ink, because transparent, does not obliterate material overwritten, and th 
notations made can be transient or permanent as the user wishes. 

Notes and commentary can be not only in written form, but auditory as wd 
The user can employ the stylus and tablet to indicate an "anchorpoint" on th 
Dataland surface for to-be-recorded voice notes. The notes are played back 
sut sequently upon coming within some critical range of the site of the 
recording. 

Sound can be itself a data type as well as a vehicle for annotations or navigj 
tional aid. An instance of sound as data in our initial SDMS prototype was 
digitally stored speech of Jerome Wiesner, Presiden! of MIT. A portion of thi 
speech was activated and played back upon approach to an image of Dr. 
Wiei ner, the volume increasing as one drew nearer by zooming in, or fading to 
the le or right as one withdrew from face-center toward the right or left, 
respective!] The speech sound was, in effect, synchronized with the location 
of the speech 

An octophonic sound capability permits the creation in the media room of 
veritable "cocktail party" of sounds and voices, any sound so re-created as to 
giv it a spatially definite apparent source in the room. Endowing sound 
sources wid specific locales effectively creales a set of "channels," aiding 
separation and iden 
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tification of a number of simultaneously heard voices. The possibility of "brows- 
ing" through sets of auditory messages, redundandy cued in visual graphics, 
can make sound as a data type or means of annotation a rich offering to the 
user. 

The ensemble of data types and associated interactions enumerated 
here form an initial set and by no means exhaust the long-term potentialities of 
user-system interaction. Before we turn to a consideration of what the future 
may hold, however, let us set down some of our major findings thus far about 
using space to organize data. 



what we learned 

A major lesson of our experience in attempting to use space to articulate 
infor-mation was the virtue of straightforward simplicity in the presentational 
aspects of the user's "virtual" spatial world. 



The system as outlined above did not spring into existence all at once but 
re-sulted from many years of Office of Naval Research research in television- 
based computer graphics and an initial version of a spatial data-management 
system, SDMS I, developed over the project's first year of DARPA support. 

SDMS I was a "2-1/2-D" spatial world: a set of 2-D data planes, so linked as 
to define a hierarchical, "laminar" space. The top level of this structure was 
Data-iand itself: a large data plane with "Items" of various sorts arranged on its 
surface, some grouped on common background colors. As described earlier, 
the large screen in front of the user functioned as a "window" onto Dataland, 
which ap-peared in its entirety on the small monitor. 

Importandy, the scheme of the data space in SDMS I included a system of 
"ports" on any data plane or layer, which, when zoomed in on, would give way 
to a next data layer, seemingly in back of or "beneath" it The information 
con- 
tained on the lower planes was logically subordinate to the topic at the port 
through which the user dove.... 

A critical focus of further research involves a set of themes having to do with 
the active engagement of the user in the environmental space of the interface, 
ex-panding the spatial domain to encompass the real-space characteristics of 
user ac-tions. The overall intention is to render the system cognizant of how 
the user is Operating in his personal space, as this space intersects with the 
shared space of the interface: directionalities in looking, speaking, touching, 
gesturing, together system awareness of user position and proximities. 
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Among these themes is the awareness of where the user is looking. Eye 
coi tact is an important adjunct in human interpersonal exchange and, in 
general, prime indicator of the focus of active awareness. 

The opportunity here is for, among other things, an eye-controlled "virtii 
camera," that is, a display that points, pans, zooms, travels, as a function of 
whei the user is looking. In the context of the imagery in SDMS so far 
discussed, pa sibilities might include even the trivial example of a spatially 
dispersed set i iteras, perhaps faces, each of which "talks" when eye- 
addressed by the user, ti directed gaze used by the system as an implicit 
"command" or act of selectioi Another example would be an image, say a map, 
which locally expands and gaij detail as a function of the point and duration of 
"looks." 



The sense of space in and at the interface includes system awareness of ti 
user near-field. The tone of appointments in the media room, especially ti 
openness of the chair, invites the user to get up out of the chair the more irma 
diately to engage himself with the data. Gesturing, arm waving, pointing, esp 
cially in conjunction with the line-of-sight indications captured throuj 
unobtrusive eye tracking, would endow the immediate space between user ai 
system with communicative meaning. 

Space-ranging techniques that permit the system to know where the user 
in the space of the interface, plus the user's body attitude, have intrinsic uses, 
system of body-borne space sensors, unobtrusively disposed about a sort of "1 
jacket" as buttons, cufflinks, epaulettes, would establish a computer-interpretal 
dot figure demarking the momentary position and attitude of the user. Amoi 
other functions, such knowledge can help to reestablish tracking contact oian < 
rant oculometer, making sensing where one is looking a more realistic propo 
tion in a dynamic environment. 

With body sensing in the user near-field, pushing Items of virtual data abe 
in a now-tangible 3-D space arises as a possibility. Or, traffic-patrol-like directi of 
items streaming by in patterns for on-the-fly perusal. The dynamics of ( 
chestration, literally like those of conducting a symphony, arise as realities, i 
pecially pertinent in an environment of spatially distinct sounds and voices in 
octaphonic surround. Pointing to virtual talkers with indications to speak, 
hurry up, to pause, becomes repertorial for the user, now "fleshed out" at Iq 
minimally in dot form in the augmented space of the interface. 

Voice direction with eye contact, a natural combination in the extra syst 

real world, persist in the realm of user-system interchange as a viable 

modal The ability to voice-activate takes on more precise meaning and 

possibilities 
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conjunction with the ability to address by voice and eye simultaneously in a co- 
ordinated fashion. To be able to point by gesture as well defines yet an addi- 
tional dimension of voice-and-eye dynamics. 

These themes, somewhat synoptically expressed, implicate actions in an ex- 
plicit space, coextensive with the real room space at the user-system interface, a 
shared space which can be artfully integrated with the virtual spaces which have 
been the focus of this, our work, so far. 

At stake here is a user-system interface, a data space, so enriched and quick- 
ened that the human becomes more than a dimensionless point, the space at the 



interface something other than empty volumes. The long-term goal is the ut- 
most in usability, where the user's natural modalities and capacities for commu- 
nicative expression meet with perceptive, complementary system response. 



rlarc Lanter 

«19» «The New Workstation: 

CD ROM 
Authoring 
Systems" 
(1986) 



Marc Canter. © Chip Simons. 

"Authoring software 
should aim to shorten 
the feedback loop 
between the computer 
and the user— between 
the idea jtselfand its 
actualization. The 
overall outcome is a 
more direct comedion 



to creativity for the 
user. " 
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« Maro Canter emerged in the 1980s as an amalgamaron of opera singer, rock 
musician, software programmer, and entrepreneur. With academic credentials 
from Oberlin College in intermedia art and electronic music, and after a brief 
stint producing music videos in New York, Canter applied his unusual array of tal- 
ents to engineering the first commerdal multimedia authoring system. He 
launched his software company, Macromind, in 1984, when the Apple Macintosh 
computer made the GUI and its potential for hypermedia applications widely 
available. Later rechristened Macromedia, its first producis, SoundWorks and 
VideoWorks, introduced multimedia production to the desktop computer. In 
1988, after the first color Macintosh appeared, Canter released the now ubiq- 
uitous Director authoring software. What followed was an explosion of newcre- 
ative possibilities, along with a good deal of Madison Avenue-style hype. By the 
ciose of the decade, desktop multimedia grew into a global phenomenon, with 
Canter at the center of the excitement, transforming the studios of artists, ar- 
chitects, and designers, reinventing the classroom, and altering the business plans 
of executives from Silicon Valley to Singapore. 

Canter's pioneering approach to multimedia authoring tools became the 
standard for al I forms of new media development. At the core of his approach 
was a notational system that looks quite similar to a musical score, an intuitive 
format that could be used easily by the artist. Canter saw the digital artist of the 
future as a "composer" of al I forms of media, orchestrating fragments of 
graphics, an-imation, and text, in juxtaposition with sound and musical passages, 
into a single artwork. His authoring system made use of "action codes" derived 
from Alan Kay's Smalltalk language, giving artists without extensive programming 
ex-perience the ability to incorporate hyperlinks into their creations by 
manipulat-ing icons on the computer screen.The metaphors Canter chose to 
describe this system have become widely adopted, including his meaning for the 
terms stage, cast, and score. This predisposition toward theater and music belies 
Canter's roots in live performance, and reinforces his vision that desktop 
multimedia would evolve into the digital Gesamtkunstwerk. » 



As the reality of CD ROM draws nearer, so does the need for development 
tools that will help programmers and artists to create CD applications fast and 
ef-ficiently. 

Currently, most CD ROM applications are text-based, displaying mainly 
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words and numbers on the screen. But as the technology moves forward, grapl 
ics and music will be stored on CD ROMs, resulting in a brand-new generada 
ofapplications. 

The tools we use in developing such applications will play an importan! rol in 
making these projects feasible. As we begin to conceive of complex new us« for 
CD ROM, the case of use and versatility of our tools will make or break ou 
concepts from day one. 

Among the tools necessaiy for these tasks, authoring systems will probabl be 
the most importan!. Integral to these systems will be a powerful, easy-to-us 
notational system that will unify entire multimedia systems on one scorejust a an 
orchestra is unified by the symphonic score used by its conductor. Thes 
scores will be capable of representing any sort of data, including the "actioi 
codes" necessary for interactive programming (or authoring). 

No longer will it be necessary to "program" a sequence of data or to devis 
another "search" algorithm. Authoring systems provide the necessary function to 
nonprogrammers to control the flow of the program, the level of interactiva and 
the production of the data itself. 

For the purposes of this discussion, we will concentrate on an authoring sys 
tern for text, art, animation, and music. It will require the simulation of a fairi 
complex output device (a premastering system) and a very fast (as real-time a 
possible) development system. This does not mean that this system will not b 
able to work with text-based CD applications. It just means that the technolog of 
the future offers us a lot more than just static text, and we must be ready to ere ate 
and control new media as they become available. 

In the following discussion, "user" refers to the user of the authoring systen 
and "end user" refers to the actual consumer in the home or office using the fina 
product. 



what Is an authoring system? 

An authoring system is a set of hardware and software tools for designing Inter 
active programs. The hardware makes it possible to convert Information into j 



machine-readable formal; the software makes it possible for nonprogrammer 
(which will often be artists and musicians) to create complex programs by defin 
ing decision points, branches, and subroutines. 

Authoring systems originated in the mainframe and minicomputer world but 
many have begun to appear for microcomputers in the past few years. Thi 



"The New Workstation: CD RDM Authoring Systems" 201 

more sophisticated authoring systems of yesterday and today produce 
compli-cated logical flowcharts that programmers use to implement their 
interactive programs. Many of these programs have "intelligent front ends" 
that ask the users questions to help them develop their programs, like "What 
sort of question would you like to ask?" or "How many dialog boxes would you 
like and what will they look like?" Using a development system based on this 
sort of authoring "language" often takes months, and sometimes years, to 
produce interactive videodisks or training programs. 

The real problem with these types of authoring systems is that they are 
very hard to use and they do not produce a final product. They produce only 
an out-line: a structure for a programmer to follow. And they are often entirely 
text-based, making them ideal for text-based applications but not for 
graphically oriented or video-based applications. 

Any language or program that can control events through time is an 
authoring system. Computer-controlled lighting systems, slide shows, and 
video edit-ing are all authoring systems in that (1) they can control sequences 
and events through time, (2) these sequences can be edited or changed, and 
(3) these sequences can be saved and retrieved from disc. 

Authoring systems have crept into the entertainment world as a form of 
automation, making the director's Ufe easier by guaranteeing that certain 
actions will lappen, no matter what. This sort of automation may have cost a 
number of op-;rators'jobs, but on the whole it has greatly enhanced the 
quality of special ef-ects in the past few years. 

Most of these large expensive systems (including the ones controlling TV 
stu-lios) are text-based, with the operator/programmer using a CRT to edit 
sequences, in conjunction with dedicated buttons, switches, and levers. But 
the ncreased interest in graphics and graphic interfaces is spawning a new 
genera-on of authoring systems that are graphically based. 

The MacroMind Sound Vision™ authoring system is graphically based and 
ses a notational "scoring" system for controlling the flow of the program 
irough time. It has no language at all but uses the notational system to 



represent jquential, branching, or even simultaneous events that occur in 
the program, 'he events may be a combination of text, graphics, music, or 
animation. This sort f system is a multimedia authoring system, since it can 
deal with all sorts of ita, not just text. 

On SoundVision's score, time moves horizontally from left to right, with 
ultiple channels of information stacked up vertically. These channels repre- 
nt the text, graphics, or music stored in the score. 



202 narc Lamer 

The scoring system is used by programmers or artists to create sequences 
of information that can easily be edited or transformed. This replaces the 
tradi-tional means of programming: typing text into a document and compiling 
it. All the data represented on the score of the authoring system can be 
edited at any time. In other words, there is no source code or compiler to 
convert the code into executable instructions. Any text or piece of artwork can 
be edited or changed even after the application is finished. 

The notational system is the unifying element in the authoring system. Text, 
graphics, animation, and music can all be synchronized and edited on the 
score. The "action codes," or programming codes, necessary for creating the 
interac-tive programs can be embedded into the score and edited just like 
any piece of animation or music. Data generated from any word processor, 
paint program, or MIDI (musical instrument digital interface) sequencer can be 
used in the Sound-Vision system. 

Simulating the programmed interaction is another necessary part of the 
authoring process and is achieved via a software/hardware system called a 
"premastering" system. These premastering units are very expensive, but 
they make it possible to test the applications before committing them to disc or 
ROM. Typ-ically, an entire project would have one, maybe two, premastering 
units to use as simulators. 



creating the applications 

Education and training applications are certain to become more common on 
optica! disc once the production costs can be brought down by using authoring 
sys-tems and other development tools. Their existence in schools and 
industry will undoubtedly change the way educators develop curricula, 
perhaps giving rise to new professions within the educational arena. 

3. By being able to customize educational courseware, regional needs can 
becatered to and accelerated or disabled learners can concentrate on spe- 
cialized material. 



4. Educat 
ors can focus on a particular subject by designing a custom lesson 
around that subject. 

5. Current 
news can be explained by integrating recent video footage fromTV with 
encyclopedia references to that area of the world or to that worldleader. 
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• Questions relating to the subject matter can be placed on the screen, and 
each answer associated with a particular screen "button." Behind each but- 
ton is an action, such as "Jump to an animated sequence for 10 seconds and 
then return" or "Jump to another document to hear the company theme 
song and exit" or "Jump to another screen of questions, answer one of 
them, and branch depending on what answer is given." 

These sorts of applications are typical of what an authoring system can create. It 
is very important to be able to model or try out some of these interactive ideas 
before committing them to disc, and an authoring system is designed to do just 
that: model an application. 

Because the people who come up with these interactive ideas are usually not 
programmers, it is equally important that they are able to model these 
environments with as little effort and ability as possible. Since teachers, trainers, 
mar-keting people, technical writers, and managers will probably be the biggest 
users of authoring systems, the systems must adhere to easy-to-learn user 
interface standards, such as those found in the Macintosh or MS-Windows 
environments. 



equipment used in an authoring system 

The workstation for a CD ROM authoring system would vary according to the 
actual work being done there. Each workstation will specialize in some function 
and thereby require different types of equipment, though very likely some equip- 
ment will be common to every workstation. The basic workstation will include: 

3. An IBM AT (or better) with 4M of memory and a 30M hard disk 
4. 

A premastering system to simulate an output device (or access to it) 
5. 

A mouse or tablet along with a full-feature keyboard 



6. 



A network of other workstations, and 2400-baud modem Une 

A text entry workstation would include optical scanners, voice recognition 
hardware, and monochrome monitors. Besides a host of text processors, the 
software available in a workstation like this would integrate data from various 
sources: standard word processors, distant databases via phone lines, ASCII, 
SYLK, dBase, and other standardized data protocols. This would all be neces- 
sary to facilitate the task of entering in hundreds of thousands of words per day. 
A graphics workstation would include video digitizers, tablets, high- 
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resolution scanners, and so on. There should also be a photo and video 
studio with camera and lighting equipment at the disposal of the artists. High- 
quality color monitors must be standard equipment in a graphics 
workstation. 

Paint programs of every make, color, and size would be standard software in 
the graphics workstation. Some of the other necessary software "equipment" 
would include video digitizing and touch-up, graphics processing (like skew, ro- 
tate, and distort), optical scanning, and 3D graphics generators. Other paint 
programs and graphics databases will be compatible with this system. 

In this sort of environment, with the right management, artists could crank 
out a meg of art a day (1 ,024,000 bytes). 

A music or sound effects workstation will require high-quality audio gear 
(speakers, tape recorders, amplifier, preamplifier) for both recording and play- 
back. Patch bays, MIDI keyboards, processing equipment, sound digitizers, 
mixers, and microphones should also be standard equipment. Color 
monitors for displaying composing programs are also necessary. 

Special facilities should be prepared with sound dampening, especially in 
the workstations designated as "audio studios." Each workstation will be 
equipped with sequencer and composition software with a variety of notational 
scoring sys-tems available, e.g., Conventional Music Notation and the 
MusicWorks grid system. Sound effects generation through algorithmic control 
will probably also be desired, as well as sampled sound editors. 

A multimedia scoring system (like SoundVision) will be used to synchronize 
the music or sound efFects to the animation. Access to other music databases, 
syn-thesis patches, or algorithms will also be possible with this workstation. A 
10-minute composition could be produced in a day at one of these 
workstations. 

An animation workstation requires much the same equipment as the 
graphics workstation, with some differences. Multiple monitors can facilitate 
the si-multaneous viewing, previewing, and editing of data. Videotape 
recorders, video SEGs (or mixers), and audio equipment are also important 



to animators who wish to transfer their work onto other media (VHS, Beta, 
3/4-inch tape). 

The animation workstation of an authoring system is where all the elements 
of the system are brought together and incorporated into the score. This is 
where the text and artwork are "animated," the music is synchronized, and 
the action codes are embedded to control the flow of the interactive 
programs. 

The SoundVision scoring system is the main software element of an 
animation workstation like this. Other types of development tools could be 
used to integrate incompatible data or to configure the premastering system, 
but creating and editing documents on the score would be the main task of 
an animator. 
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So you can see there is more thanjust one type of workstation involved in 
CD ROM project. Managers, designers, and executives could theoretically 
?ork in an animation workstation. Arusts could roam between graphics and an- 
nation stations, and producers and directors would have to know them all 

qually well. 



fAILORING YOUR WORKSTATION 

lach workstation should be customized to the particular project and application 
icing produced. One of the best ways to utilize workers' abilities would be to use 
emote workstations. You could place them anywhere (especially at home) so that 
fork could continue seven days a week and on holidays. And be fun! 



IOW TO USE AN AUTHORING SYSTEM \fage One 

The first stage in using an authoring system is to block out the approximate 
imings and interaction desired in the application. Atypical application would iave 
a menu at the beginning, at least two or three other menus somewhere else, nd 
some sort of ending section. 

Text and artwork (which should have been created earlier), along with aeon 
codes, are placed into a document's score at the approximate locations. Di-log 
boxes, menus, and buttons, which start off as artwork, soon turn into iteractive 
controls when an action code is associated with them. The connec-.on between 



artwork and action code is all done on the score, at the exact »ame desired. 
(The exact logistics of how to assign an action code in the score i explained 
below.) 

Once a rough draft has been worked out, initial viewings can be used to de-:ct 
mistakes, wrong codes, or ill-conceived notions. This process of trying things ut as 
soon as possible allows the author to correct any grievous errors before ley 
become uncontrollable. 

The material can also be edited at any time, so different sections of the pro-;ct 
can be polished at diherent times, and conceivably by difFerent people. 
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Stage Two 

The second stage in using an authoring system (once the timings have beei 
frozen) is to add more action codes. These codes may link several documents 
to gether, branch to other sections and return, or even start other 
applicationg Music and animation can also be added at this stage of the 
development procesa 

Memory limitations and access times are some of the typical problems en 
countered when developing applications with authoring systems. Once a write 
releases the tremendous potential of the system, he usually goes overboard 
anc asks the system to do too much. For instance, jump from a section on 
DNA syn thesis, to a musical selection by Beethoven, to a survey of Picasso 
paintings, al in 0:15 second, and then return to the chemistry section and 
continue the lesson 

Authoring systems can help organize and control a huge amount of data, bu 
it is very easy to overextend yourself. Early on in your design you must determiru 
how much data is enough. Just how many sequences of animation 
demonstrat ing the principles of fluid mechanics or wave dynamics will be 
suflicient to get tru point across? Whatever you decide, it is very important to 
not be too ambitious Tackle only a small part of a huge task at a time. There 
will always be room foi more later. 

It's important to realize that authoring systems will evolve and grow witj 
technology. In the future, new types of authoring systems will be designed to 
taki advantage of new technology. Besides simple menu choices and passive 
viewing new standards of interacting will develop, such as text entry or real- 
time input. 

•/' 

Stage Three 

The final stage of using an authoring system is the debugging of the 
application Do all the questions make sense? Does the program flow smoothly? 



Are there air] interactions that don't go anywhere? Is the artwork less than 
perfect? Goinj back to the score and changing action codes is as easy as 
editing text or touch ing up graphics. 

This stage is probably the most important one, as it's always that last 5% a 
an application that can really make or break it. 

Because of the nature of authoring systems, applications can evolve an« 
change as they are worked on. Sections that make perfect sense on paper 
oftei turn out to be less than expected. Because of this, an authoring system 
use should always keep an open mind, ever ready to shift gears and do things 
differ 
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A good authoring system should facilitate this need by making it as easy as 
jossible to edit and change the application at any time 



Important issues for CD ROM authors 



Authoring software should aim to shorten the feedback loop between the 
com-juter and the user — between the idea itself and its actualization. The 
overall out-:ome is a more direct connection to creativity for the user and a 
higher level of >roductivity. 

This type of interaction is exactly what computers were made for: real-time 
ools. An example of this real-time interaction can be seen in Music Works, where 
he notes are heard as they are placed onto the score, or in VideoWorks, 
where ou can edit an image as you watch it animate. 

Widespread use of such authoring workstations will enable large groups of 
rtists and programmers to produce very large databases of text, animation, 
andlusic. By keeping the notational system as generalized as possible, 
workstationske this have the potential of controlling Broadway productions or 
entire televi-lon studios 



Tim Berners-Lee 
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Lee. 



Mr? important pari.... is the integration ofa hypertext system with existing 
data, so as to provide a universal system, and to achieve critical usefulness.. 
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« In the early 1980s, the British engineerTim Berners-Lee began to develop a 
computer system for the electronic publishing of project management notes at 
CERN, the partiole physics laboratory jn Geneva ; Switzerland. This system, 
named Enquire, was to have enabled the storing, retrieval, and hyperlinking of 
documents. It was never completed, but Berners-Lee expanded on its underlying 
:oncepts to explore how a hypertext system might work in conjunction with the 
Internet.The result js the unassuming proposal reprinted here,which is nothing 
less than a prospectus for the creation of the World Wide Web. 

Berners-Lee's colleagues at CERN largely ignored his proposal, however. 
Working under his own initiative, jn the fall of 1990 Berners-Lee completed the 
First Web browser and server software. In 1991, he began to distribute his 
software, now named the World Wide Web, to scientists over the Internet. After 
two years of gradual adoption among scientific colleagues, who, like Berners- 
Lee, used the Next computer, the Web began to take off when Marc Andreesen 
and a group of graduate students at the University of Illinois, Campagne- 
Urbana, adopted the browser for the Macintosh and PC. In 1993, their version of 
the browser, Mosaic, was widely distributed on the Internet, transforming the 
World Wide Web into a mass medium. 

Berners-Lee's Web is a software system that unites research, documents, 
programs, laboratories, and scientists jn a fluid, open, hypermedia environment. 
He proposes a decentralized network to which new servers can be added at any 
time without the approval of a centralized authority. Berners-Lee's Web is in- 
herently dynamic, capable of expanding at an explosive rate; this was a signifi- 
cant departure from the hierarchical data systems that had previously been the 
standard. Another cornerstone of the system is the user's ability to annotate 
documents himself— to write his own "private links," as Berners-Lee phrases it 
here, creating an infinitely expanding variety of connections. While this paper 
ad-dresses only how the Web might benefit CERN, Berners-Lee was well aware 
of his system's potential to link documents across the globe, and to transform our 
information culture. It js worth noting that while Berners-Lee focuses here on 
hypertext, he also addresses the potential to incorporate sound and image. From 



the start he saw the Web's eventual embrace of multimedia, which could well 
prove to be its enduring legacy. » 
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THIS proposal concern! the management of general information about acceler- 
ators and experiments at CERN. It discusses the problems ofloss of information 
about complex evolving systems and derives a solution based on a distributed hy- 
pertext system. 



overview 

Many of the discussions of the future at CERN and the LHC era end with the 
question — "Yes, but how will we ever keep track of such a large project?" Thu 
proposal provides an answer to such questions. Firstly, it discusses the 
problem of information access at CERN. Then, it introduces the idea of 
linked infor mation systems, and compares them with less flexible ways of 
finding informa-tion. 

It then summarises my short experience with non-linear text systems knowi 
as "hypertext", describes what CERN needs from such a system, and what 
in-dustry may provide. Finally, it suggests steps we should take to involve 
ourselva 

with hypertext now, so that individually and collectively we may understand 
what we are creating. 



losing information at CERN 

CERN is a wonderful organisation. It involves several thousand people, many 
of them very creative, all working toward common goals. Although they are 
nomi-nally organised into a hierarchical management structure, this does not 



constrain the way people will communicate, and share information, 
equipment and software across groups. 

The actual observed working structure of the organisation is a multiply con- 
nected "web" whose interconnections evolve with time. In this environment, a 
new person arriving, or someone taking on a new task, is normally given a 
few [lints as to who would be useful people to talk to. Information about what 
facil-jties exist and how to find out about them travels in the corridor gossip 
and ocasional newsletters, and the details about what is required to be done 
spread in i similar way. All things considered, the result is remarkably 
successful, despite Occasional misunderstandings and duplicated effort. 
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A problem, however, is the high turnover of people. When two years is a typ- 
cal length of stay, information is constantly being lost. The introduction of the 
lew people demands a fair amount of their time and that of others before 
they lave any idea of what goes on. The technical details of past projects are 
some-imes lost forever, or only recovered after a detective investigation in an 
emer-jency. Often, the information has been recordad, it just cannot be 
found. 

If a CERN experiment were a static once-only development, all the informa- 
ion could be written in a big book. As it is, CERN is constantly changing as new 
deas are produced, as new technology becomes available, and in order to 
get round unforeseen technical problems. When a change is necessary, it 
normally ffects only a small part of the organisation. A local reason arises for 
changing a art of the experiment or detector. At this point, one has to dig 
around to find out 'hat other parts and people will be affected. Keeping a book 
up to date becomes npractical, and the structure of the book needs to be 
constantly revised. 

The sort of information we are discussing answers, for example, 
questions ke 



19. Where is this module used? 
Who wrote this code? Where does he work? 
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20. What documents exist about that concept? 

21. Which laboratories are included in that project? 

22. Which systems depend on this device? 

23. What documents refer to this one? 

The problems of information loss may be particularly acute at CERN, but in 
this case (as in certain others), CERN is a model in miniature of the rest of 
world in a few years' time. CERN now meets some problems which the rest of 
the world will have to face soon. In 10 years, there may be many commercial 
solutions to the problems above, while today we need something to allow us 
to continue. 



linked information systems 

In providing a system for manipulating this sort of information, the hope would 
be to allow a pool of information to develop which could grow and evolve witl 
the organisation and the projects it describes. For this to be possible, 

the method of storage must not place its own restraints on the 
information. 

This is why a "web" of notes with links (like references) between them is fa 
more useful than a fixed hierarchical system. When describing a complex 
system many people resort to diagrams with circles and arrows. Circles and 
arrows leavi one free to describe the interrelationships between things in a way 



that tables, fo example, do not, The system we need is like a diagram of 
circles and arrowa where circles and arrows can stand for anything. 

We can cali the circles nodes, and the arrows links. Suppose each node is 
lik a small note, summary arricie, or comment. I'm not overly concerned here 
witl whether it has text or graphics or both. Ideally, it represents or describes 
one pai ticular person or object. Examples of nodes can be 

11. People 

12. Software modules 

13. Groups of people 

14. Projects 

15. Concepts 

16. Documents 

17. Types of hardware 

18. Specific hardware objects 
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The arrows which link circle A to circle B can mean, for example, that A 
depends on B 

24. ispartofB 

25. made B 

26. refers to B 

27. uses B 

28. isan example ofB 

These circles and arrows, nodes and links, have different significance in var- 
3 sorts of conventional diagrams: 



GRAM 


NODES ARE 


ARROWS MEAN 


lily tree 


People 


"Is parent of 


iflow diagram 


Software modules 


"Passes data to" 



icndency 


Module 


"Depends on" 


LT chart 


Tasks 


"Must be done 






before" 


anisation 


People 


"Reports 


al chart 




to" 



The system must allow any sort of information to be entered. Another per- 
must be able to find the information, sometimes without knowing what he is 
ing for. 

[n practice, it is useful for the system to be aware of the generic types of the s 
between items A dependences,£or example), and the types of uodes (peopVe, 
igs, documents ...) without imposing any limitations. 

u Problem with Trees 

iny systems are organised hierarchically. The CERNDOC documentation sys- 
i is an example, as is the Unix file system, and the VMS/HELP system. A tree 
> the practical advantage of giving every node a unique name. However, it 
does t allow the system to model the real world. For example, in a 
hierarchical CLP system such as VMS/HELP, one often gets to a leaf on a 
tree such as 



HELP COMPILER SOURCE_FORMAT PRAGMAS DEFAULTS ly to find a 
reference to another leaf: "Picase see HELP COMPILER COMMAND 
OPTIONS DEFAULTS PRAGMAS" 
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and it is necessary to leave the system and re-enter it. What was needed was a 
link from one node to another, because in this case the Information was not 
naturalh organised into a tree. 

Another example of a tree-structured system is the uucp News system (try 
"m" under Unix). This is a hierarchical system of discussions ("newsgroups'"), 
each containing ardeles contributed by many people. It is a very useful 
method of pooling expertise, but suffers from the inflexibility of a tree. Typically, 
a dis-cussion under one newsgroup will develop into a different topic, at which 
point it ought to be in a different part of the tree. (See Figure 1). 

The Problem with Keywords 

Keywords are a common method of accessing data for which one does not 
have the exact coordinates. The usual problem with keywords, however, is 
that two people never choose the same keywords. The keywords then become 
useful onty to people who already know the application well. 

Practical keyword systems (such as that of VAX/NOTES for example) re- 
quire keywords to be registered. This is already a step in the right direction. 

A linked system takes this to the riext logical step. Keywords can be nodea 
which stand for a concept. A keyword node is then no different from any othei 
node. One can link documents, etc., to keywords. One can then find 
keywordj by finding any node to which they are related. In this way, documents 
on similai topics are indirecdy linked, through their key concepts. 

A keyword search then becomes a search starting from a small number o! 
named nodes, and finding nodes which are ciose to all of them. 

It was for these reasons that I first made a small linked informatioi 
system, not realising that a term had already been coined for the idea: hyper 
text. 



A solution: hypertext 



Personal Experience with Hypertext 



In 1980,1 wrote a program for keeping track of software with which I was in 
volved in the PS control system. Called Enquire, it allowed one to store snippet 
of information, and to link related pieces together in any way. To find information 



jmm iiiaiiuii i idndyemenr:« rroposai 215 



From mcvax!uunet!pyrdc!pyrn j!rutgers!bellcore!geppetto!duncaa Thu 
Mar.,. 

Article 93 of alt.hypertext: 
Path: 

cernvax!mcvax!uunet!pyrdc!pyrnj!rutgers!bellcore!geppetto!duncan 

>From: duncan8geppetto.ctt.bellcore.com (Scott Duncan) 

Newsgroups: alt.hypertext 

Subject: Re: Threat to free information networks 

Message-ID: <14646@bellcore.bellcore.com> 

Date: 10 Mar 89 21:00:44 GMT 

References: <1784.2416BB47Sisishq.FIDONET.ORG> <3437?uhccux. 
uhcc . . . 

Sender: newsSbellcore.bellcore.com 
Reply-To: duncan@ctt.bellcore.com (Scott Duncan) 
Organization: Computer Technology Transfer, Bellcore 
Lines: 18 



Doug Thompson has written what I felt was a thoughtful article on 
censorship-my acceptance or rejection of its points is not particularly 
germane to this posting, however. 

In reply Greg Lee has somewhat tersely objected. 

My question (and reason for this posting) is to ask where we might 
logically take this subject for more discussion. Somehow alt.hypertext 
does not seem to be the proper place. 



Would people feel it appropriate to move to altindividualism or even one 
of the soc groups, lam not so much concerned with the specific issue 
of censorship of rec.humor.funny, but the views presented in Greg's 
article. 

Speaking only for myself, of course, lam ... 
Scott P. Duncan (duncanfcttbellcore.com OR... 
Ibellcorelcttlduncan) 

(Bellcore, 444 Hoes Lane RRC 1H-210, Piscataway, NJ . . 
(201-699-3910 (w) 201-463-3683 (h)) 



igure 1. An article in the UUCP News scheme. The Subjectfield allows notes on the 
same topic be linked together within a "newsgroup". The name of the 



newsgroup (alt.hypertext) is a '.erarchical name. This particular note expresses 
a problem with the strict tree structure of the heme: this discussion is related to 
several areas. Note that the "References," "From"and "Subject" '.Ids can all be 
used to generate links. 

ie progressed via the links from one sheet to another, rather like in the oid 
imputer game "adventure." I used this for my personal record of people and 
odules. It was similar to the application HyperCard produced more recently by 
for the Macintosh. A difference was that Enquire, although lacking the 



Documentation of the RPC project (concept) 

Most of the documentation is available on 
VMS , with the two principle manuals being 
stored in the CERNDOC systera. 

29. includes: The VAX/NOTES conference 
VXCERN: :RPC 

30. includes: Test and Example suite 

31. includes: RPC BUG LISTS 

32. includes: RPC System: 
Implementation Cuide 
Information for maintenance , 
porting , etc . 

33. includes: Suggested Development Strategy 
for RPC Applications 

34. includes: "Notes on RPC", Draft 1, 20 feb 
86 

35. includes : "Notes on Proposed RPC 
Development" 18 Feb 86 

36. includes: RPC User Manual 

How to build and run a distributed system. 
9) includes : Draft Specifications and 
Implementation Notes 



20. describes: THE REMOTE PROCEDURE CALL 
PROJECT in DD/OC 

Help Display Select Back Quit Mark Goto hiark 
Link Add Edit 

FIGURE 2. A screen in an Enquire scheme. 

fancy graphics, ran on a multiuser system, and allowed many people to access 
di same data. 

This example is basically a list, so the list of links is more important than di 
text on the node itself. Note that each link has a type ("includes," for example 
and may also have comment associated with it. (The bottom line is a menu 
bar 
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ISSi 




Soon after my re-arrival at CERN in the DD division, I found that the env 
ronment was similar to that in PS, and I missed Enquire. I therefore produced 
version for the VMS, and have used it to keep track of projects, people, groupi 
experiments, software modules and hardware devices with which I have worket 
I have found it personally very useful. I have made no effort to make it suitabl 
for general consumption, but have found that a few people have successful) 
used it to browse through the projects and find out all sorts of things of their o\» 
accord. 

Hot Spots 

Meanwhile, several programs have been made exploring these ideas, both 
con mercially and academically. Most of them use "hot spots" in documents, 
ul icons, or highlighted phrases, as sensitive areas. Touching a hot spot 
with mouse brings up the relevant information, or expands the text on the 
screen I 
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include it. Imagine, then, the references in this document, all being 
associated with the network address of the thing to which they referred, so 
that while read-ing this document you could skip to them with a click of the 
mouse. 

"Hypertext" is a term coined in the 1960s by Ted Nelson, 1 which has become 
popular for these systems, although it is used to embrace two different ideas. 
One idea (which is relevant to this problem) is the concept: 

"Hypertext": Human-readable information linked together in an uncon- 
strained way. 

The other idea, which is independent and largely a question of technology 
and time, is of multimedia documents which include graphics, speech and 
video. I will not discuss this latter aspect further here, although I will use the 
word "hy-permedia" to indicate that one is not bound to text. 

It has been difficult to assess the eftect of a large hypermedia system on an 
or-ganisation, often because these systems never had seriously large-scale 
use. For this reason, we require large amounts of existing information should 
be accessi-ble using any new information management system. 



CERN requirements 

To be a practical system in the CERN environment, there are a number of 
clear practical requirements. 

Remote access CERN is distributed, and access from remote 

across networks machines is essential. 
Heterogeneity Access is required to the same data from 

different types of systems (VM/CMS, Macintosh, 

VAX/VMS, Unix). 
Non-centralisation Information systems start small and grow. 

They also start isolated and then merge. A new 

system must allow existing systems to be linked 



together without requiring any central control or 
coordination. 

Access to existing data If we provide access to existing databases 
as 

though they were in hypertext form, the system will 
get offthe ground quicker. This is discussed further 
below. 

Private links One must be able to add one's own private 



218 Tim Berners-Lee 

Bells and whistles 

links to and from public information. One must also be able to annotate links, 
as well as nodes, privately. 

Storage of ASCII text, and display on 24x80 screens, is in the short term 
sufficient, and essential. Addition of graphics would be an optional extra with 
very much less penetration for the moment. 

Data Analysis 

An intriguing possibility, given a large hypertext database with typed links, is 
that it allows some degree of automatic analysis. It is possible to search, for 
ex-ample, for anomalies such as undocumented software or divisions which 
contain no people. It is possible to generate lists of people or devices for other 
purposes, such as mailing lists of people to be informed of changes. 

It is also possible to look at the topology oian organisation or a project, and 
draw conclusions about how it should be managed, and how it could evolve 
This is particularly useful when the database becomes very large, and groups 
a projects, for example, so interwoven as to make it difficult to see the wood for 
th< trees. 

In a complex place like CERN, it's not always obvious how to divide peopl< 
into groups. Imagine making a large three-dimensional model, with people rep 
resented by little spheres, and strings between people who have something ii 
common at work. 

Now imagine picking up the structure and shaking it, until you make soim 
sense of the tangle: perhaps, you see tightly knit groups in some places, and 
u some places weak areas of communication spanned by only a few people. 
Perhap a linked information system will allow us to see the real structure of the 
organi sation in which we work. 



Live Links 

The data to which a link (or a hot spot) refers may be very static, or it may be 
tern porary. In many cases at CERN information about the state of systems is 
chang ing all the time. Hypertext allows documents to be linked into "live" data 
so tha every time the link is followed, the information is retrieved. If one sacrifices 
porta 
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bility, it is possible to make following a link fire up a special application, so that 
diagnostic programs, for example, could be linked directly into the 
maintenance A uide. 



Nonrequirements 

Discussions on hypertext have sometimes tackled the problem of copyright 
en-forcement and data security. These are of secondary importance at CERN, 
where information exchange is still more important than secrecy. Authorisation 
and ac-counting systems for hypertext could conceivably be designed which are 
very so-phisticated, but they are not proposed here. 

In cases where reference must be made to data which is in fact protected, 
ex-isting file protection systems should be sufficient. 



specific applications 

The following are three examples of specific places in which the proposed 
sys-tem would be immediately useful. There are many others. 



Development Project Documentation 

The Remote procedure Cali project has a skeleton description using Enquire. 
Al-though limited, it is very useful for recording who did what, where they are, 
what documents exist, etc. Also, one can keep track of users, and can easily 
append any extra little bits of informadon which come to hand and have 
nowhere else to be put. Cross-links to other projects, and to databases which 
contain information on people and documents, would be very useful, and save 
duplication of information. 



Document Retrieval 

The CERNDOC system provides the mechanics of storing and prindng 
documents. A linked system would allow one to browse through concepts, 
docu-nents, systems and authors, also allowing references between 
documents to be itored. (Once a document had been found, the existing 
machinery could be in-roked to print it or display it.) 
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The "Personal Skills Inventory " 

Personal skills and experience are just the sort of thing which need hypertext 
fles ibility. People can be linked to projects they have worked on, which in turn 
cai be linked to particular machines, programming languages, etc. 



the state of the art in hypermedia 

An increasing amount of work is being done in hypermedia research at univei 
sities and commercial research labs, and some commercial systems have 
resultec There have been two conferences, Hypertext '87 and '88, and in 
Washingto D.C., the National Institute of Standards and Technology (NST) 
hosted a worl shop on standardisation in hypertext, a followup of which will 
occur durin 1990. 

The Communications ofthe ACM special issue on hypertext contains man 
references to hypertext papers. 2 A bibliography on hypertext is given in Pit 
ceedings ofthe Hypertext Standardisation Workshop, January 16-18, 1990; 
and a uucp newsgroup alt.hypertext exists. 4 1 do not, therefore, give a list here 

Browsing Techniques 

Much of the academic research is into the human interface side of browsin 
through a complex information space. Problems addressed are those of 
makin navigation easy, and avoiding a feeling of being "lost in hyperspace." 
Whilst dh results ofthe research are interesting, many users at CERN will be 
accessing th system using primitive terminals, and so advanced window styles 
are not so in portant for us now. 



Interconnection or Publication? 



Most systems available today use a single database. This is accessed by 
man users by using a distributed file system. There are few producis which 
take Te Nelson's idea of a wide "docuverse" literally by allowing links between 
nodes i different databases. In order to do this, some standardisation would be 
necessar However, at the standardisation workshop, the emphasis was on 
standardis; tion ofthe formal for exchangeable media, nor for networking. This 
is prompte 
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by the strong push toward publishing of hypermedia information, for example 
on optical disk. There seems to be a general consensus about the abstract 
data model which a hypertext system should use. 

Many systems have been put together with little or no regard for portabil- 
ity, unfortunately. Some others, although published, are proprietary software 
which is not for external release. However, there are severa! interesting 
projects and more are appearing all the time. Digital's "Compound Document 
Archi-tecture" (CDA), for example, is a data model which may be extendible 
into a hypermedia model, and there are rumours that this is a way Digital 
would like togo. 

Incentives and CALS 

The U.S. Department of Defence has given a big incentive to hypermedia 
re-search by, in effect, specifying hypermedia documentation for future 
procure-ment. This means that all manuals for parts for defence equipment 
must be provided in hypermedia form. The acronym CALS stands for 
Computer-aided Acquisition and Logistic Support. 

There is also much support from the publishing industry, and from librari- 
ans whose job it is to organise information. 

What Will the System Look Like? 

Let us see what components a hypertext system at CERN must have. 

The only way in which sufficient flexibility can be incorporated is to separate 
the information storage software from the information display software, with a 
well-defined interface between them. Given the requirement for network 
access, it is natural to let this clean interface coincide with the physical division 
between the user and the remote database machine. 

This division also is important in order to allow the heterogeneity which is 
required at CERN (and would be a boon for the world in general). 



Therefore, an important phase in the design of the system is to define 
this interface. After that, the development of various forms of display 
program and of database server can proceed in parallel. This will have been 
done well if many different information sources, past, present and future, can 
be mapped onto the definition, and if many different human interface 
programs can be written over the years to take advantage of new technology 
and standards. 
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accessing existing data 

The system must achieve a critical usefulness early on. Existing 
hypertext systems have had to justify themselves solely on new data. If, 
however, there was an existing base of data of personnel, for example, to 
which new data could be linked, the value of each new piece of data would be 
greater. 

What is required is a gateway program which will map an existing structure 
onto the hypertext model, and allow limited (perhaps read-only) access to it. 
This takes the form of a hypertext server written to provide existing information 
in a form matching the standard interface. One would not imagine the server 
ac-tually generating a hypertext database from an existing one: rather, it would 
generate a hypertext view of an existing database. 

Some examples of systems which could be connected in this way are 

uucp News 



VAX/Notes 



CERNDOC 



File systems 

The Telephone Book 



The uuix manual 

This is a Unix electronic conferencing system. A server for uucp news could 
makes links between notes on the same subject, as well as showing the 



structure of the conferences. This is Digital's electronic conferencing system. 
It has a fairly wide following in FermiLab, but much less in CERN. The 
topology of a conference is quite restricting. 5 This is a document registration 
and distribution system running on CERN's VM machine. As well as 
documents, categories and projects, keywords and authors lend themselves 
to representation as hypertext nodes. 6 

This would allow any file to be linked to from other hypertext documents. 
Even this could be viewed as hypertext, with links between people and 
sections, sections and groups, people and floors of buildings, etc. This is a 
large body of computer-readable text, currently organised in a fiat way, but 
which also 
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contains link information in a standard formal ("See also ..."). 



Databases A generic tool could perhaps be made to allow 

any database which uses a commercial DBMS to be displayed 
as a hypertext view. 

In some cases, writing these servers would mean unscrambling or 
obtaining details ofthe existing protocols and/or file formais. It may nol be 
praclical to pro-vide the full functionality of the original system through 
hypertext. In general, it will be more important to allow read ; ccess to the 
general public: it may be that there is a limited number of people who are 
providing the information, and that they are conteni to use the existing 
facilities. 

It is sometimes possible to en' '-.nee an existing storage system by coding 
hypertext information in, if one knows that a server will be generating a 
hypertext representation. In "news" ardeles, for example, one could use 
(in the text) a standard formal for a reference to another article. This would 
be picked out by the hypertext gateway and used to generate a link to that 
note. This sort of en-hancement will allow greater integration between oid and 
new systems. 

There will always be a large number of information management systems — 
we get a lot of added usefulness from being able to cross-link them. However, 
we will lose out if we try to constrain them, as we will exclude systems and 
hamper the evolution of hypertext in general. 



conclusion 



We should work toward a universal linked information system, in which gener- 
ality and portability are more important than fancy graphics techniques and 
complex extra facilities. 

The aim would be to allow a place to be found for any information or 
reference which one felt was important, and a way of finding it afterwards. The 
result should be sufficiently attractive to use that the information contained 
would grow past a critical threshold, so that the usefulness of the scheme 
would in turn encourage its increased use. 

The passing of this threshold accelerated by allowing large existing data- 
bases to be linked together and with new ones. 
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A Practical Project 

Here I suggest the practical steps to go to in order to find a real solution at 
CERN. After a preliminary discussion of the requirements listed above, a 
survey of what is available from industry is obviously required. At this stage, we 
will be looking for systems which are future-proof: 

3. Portable, or supported on many platforms. 
4. Extendible to new data formats. 

We may find that with a little adaptation, parts of the system we need can 
be combined from various sources: for example, a browser from one source 
with a database from another. 

I imagine that two people for 6 to 12 months would be sufficient for this 
phase of the project. 

A second phase would almost certainly involve some programming in order 
to set up a real system at CERN on many machines. An important part of this, 
discussed below, is the integration of a hypertext system with existing data, so 
as to provide a universal system, and to achieve critical usefulness at an early 
stage. 

(. . . and yes, this would provide an excellent project with which to try our new 
object oriented programming techniques!) 



George Landow and Paul Delany 

"Hypertext, Hypermedia and 
Literary Studies: The State 
oFthe Art" (1991) 



Marh Amerika, Grammatron. ©Mark 
Amerika. Courtesy ofMark Amerika. 

"Hypertext... changes oursense of 
authorship, authorial property, and creativity 
(or originality) by moving away from the 
constrictions ofpage-bound technology. In so 
doing, it promises to nave an effect on 
cultural and intellectual disciplines as 
important as those produced by earlier shifts 
in the technology of cultural memory that 
followed the invention of writing and 
printing. " 
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« George Landow has done much to pave the way for the critical acceptance of 
hypermedia as a medium for creative and academic writing. A noted literary 
theorist from Brown University — a hotbed for hypertext since Ted Nelson was a 
fellow there during the 1960s — Landow introduced the potential of hypermedia 
to a generation of writers and scholars eager to explore the medium's possibili- 
ties. His associates include noted hyperfiction authors Stuart Moulthrop and 
Mark Amerika, who began pioneering forms of interactive writing while study-ing 
hypertext at Brown. Landow also helped develop the Intermedia software sys-tem 
jn the late 1980s, one of the first authoring tools for creating interactive texts. 
Intermedia was popular among hypermedia enthusiasts at the time and served 
as a model forTim Berners-Lee's initial development of the World Wide Web. 

Hypertext erodes the rigidity of print by encouraging the reader to navigate its 
contents freely clicking through it jn multiple directions, creating a unique, 
participatory reading experience. Landow, with Paul Delany from Simon Fraser 
University, discusses hypermedia's impact on reading and writing, and how this 
ability to reassemble the text is an almost "embarrassingly literal" example of 
the deconstruction of meaning explored jn late-twentieth-century critical theory. 
Landow and Delany examine literary conventions, such as footnotes and in- 
dexes, to illustrate how electronic means of linking texts have roots jn tradi- 
tional writing practice. They describe how hypertext extends these devices in 
such a way that they become central to the reading, transforming the text into a 
complex network of paragraphs, sentences, and fragments of "textual units" or 
"lexias," as they are called The reader's ability to explore this network freely, and 
disrupt the linearity of the text, enables a dramatic shift from what they regard 
as hierarchical, centralized, and author-dominated literary forms. Landow and 
Delany describe this new form as "intertextualities," a terrain where the bound- 
aries between literary works dissolve as they join into a single, vast "docuverse." 



While they note that this literature of the future may introduce chaos into the 
experience of reading, they see a greater potential for an enhanced "technology of 
cultural memory/' inspired by Vannevar Bush and Douglas Engelbart, that calis 
for a revitalized cultural dialogue and collaborative exchange between author 
and reader. » 
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hypertext, hypermedia and the history of the text 

The written text is the stable record of thought, and to achieve this stability the 
text had to be based on a physical medium: clay, papyrus or paper; tablet, 
scroll or book. 1 But the text is more thanjust the shadow or trace of a thought 
already shaped; in a literate culture, the textual structures that have evolved 
over the cen-turies determine thought almost as powerfully as the primal 
structure that shapes all expression, language. So long as the text was married 
to a physical media, read-ers and writers took for granted three crucial 
attributes: that the text was linear, bounded, undfixed. Generations of scholars 
and authors internalized these qual-ities as the rules of thought, and they had 
pervasive social consequences. We can define Hypertext as the use of the 
computer to transcend the linear, bounded and fixed qualities of the traditional 
written text. 2 Unlike the static form of the book, a hypertext can be composed, 
and read, non-sequentially; it is a variable structure, composed of blocks of 
text (or what Roland Barthes terms lexia) and the electronic links that join 
them. 3 Although conventional reading habits apply within each block, once 
one starts to follow links from one block to another new rules and new 
experience apply. Instead of facing a stable object — the book — en-closing an 
entire text and held between two hands, the hypertext reader sees only the 
image of a single block of text on the computer screen. Behind that image 
lies a variable textual structure that can be represented on the screen in dif- 
ferent ways, according to the reader's choice of links to follow. Metaphors that 



can help us to visualise the structure "behind" the screen include a network, 
a tree diagram, a nest of Chinese boxes, or a web. 

The immediate ancestor of modern hypertext was described in a pioneering 
article by Vannevar Bush in the 1945 Atlantic Monthly. 4 Bush called for me- 
chanically linked information-retrieval machines to help scholars and decision 
makers in the midst of what was already an explosion of information. In the 
1960s Douglas C. Englebart and Theodor H. Nelson began to design and 
im-plement computer systems that could implement some of these notions of 
linked texts, and today hypertext as a term refers almost exclusively to 
computerized hypertext programs, and to the textual structures that can be 
composed with their aid. Hypertext programs began to be widely available on 
personal computers in the late 1980s; current examples are Guide and 
Linkway for IBM-compatible FCs, Intermedia for Macintoshes running A/UX, 
and Writing Space and HyperCard for most Macintoshes. 
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Because hypertext breaks down our habitual way of understanding and 
ex-periencing texts, it radically challenges students, teachers, and theorists of 
liter-ature. But it can also provide a revelation, by making visible and explicit 
mental processes that have always been part of the total experience of 
reading. For the text as the reader imagined it — as opposed to the physical 
text objectified in the book — never had to be linear, bounded or fixed. A reader 
couldjump to the last page to see how the story ended; could think of relevant 
passages in other works: could re-order texts by cutting and pasting. Still, the 
stubborn materiality of the text constrained such operations: they required 
some physical task such as flip-ping pages, pulling another book from a shelf, 
or dismembering the original text beyond repair. 5 Over the centuries, readers 
developed a repertoire of aids to textual management; these aids operated 
both within a single volume, and in the re-lations between volumes. They 
constituted a proto-hypertext, in which we can find important models for 
hypertext design today — though the special powers of the computer allow us 
to look beyond the textual aids that evolved during the long history of writing 
and printing. 

Within the individual volume of the traditional book we may find such in- 
ternal hypertextual functions as tables of contents, page-numbers, 
chapters, verses, rubrications, footnotes, and indexes. Some of these may be 
assigned by the original author, others by specialists in textual organization 
such as indexers or printers, or by later generations of scholars. External 
hypertextual functions have traditionally been post-authorial, supplied by 
librarians and bibliographers. Indeed, once one extends the idea of "a text" to 
include a collection of volumes the object of study ceases to be bounded, 



linear or fixed, and some kind of im-plicitly hypertextual organization will 
always be necessary. 

How does hypertext actually work? Two examples, already familiar to most 
scholars, will help to show its underlying principies. Take first the elementary, 
but not trivial, question faced by large research libraries: we have a hundred 
and fifty kilometers or more of shelves — in what order should the books be 
placed on them? The primitive solution is an accession system, like the 
British Library, where each new volume is simply added on at the "end" of the 
shelf, and volumes are classified by the physical address of their original 
placement. Modern systems try to establish a logical order that places related 
books together, independent of any particular arrangement of shelves. This 
creales a kind of hypertextual link-ing, but done on a fixed and one- 
dimensional basis (so that, for example, one must choose between uniting 
all the books on the same subject, or all by the same author). 

On-line library catalogues, now coming into general use, can be thought 

of 
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as coarse-grained and rudimentary hypertext systems that support a "virtual" 
re-arrangement and retrieval of individual volumes at the terminal. However, 
they are usually limited to a few standardi/ed search categories, such as 
author, title and subject; and they cannot discriminate between any textual 
units smaller than a complete book. More sophisticated systems, such as the 
on-line MLA Bibli-ography, can perform subtler searches, using more detailed 
descriptors and deal-ing with articles as well as books. But neither deserve to 
be called true hypertext systems, because they operate on textual 
classifications rather than on the actual underlying complete texts. True 
hypertext must be able to define textual units, and link them in various ways, 
within an overall textbase or, to use another term now gaining currency, 
"docuverse." 

Our second example of the principies underlying hypertext could be any 
standard scholarly article in the humanities. In reading an ardele on, say, 
Joyce's Ulysses, one reads through the main text, encounters a symbol that 
indicates the presence of a footnote, and leaves die main text to read that note, 
which can con-tain a citadon of passages in Ulysses that supposedly support 
the argument in question as well as information about sources, influences, 
historical background, or related ardeles. In each case, the reader can follow 
the link to another text and thus move endrely outside the scholarly article 



itself. Having completed reading the note — and perhaps some of the texts to 
which it refers — one returns to the main text and continues reading until one 
encounters another note, and again leaves the main text. 
This kind of reading constitutes a mental model of hypertext. Suppose now 
that one could simply touch the page where the symbol of a note, reference, 
or annotadon appeared, and that act instantly brought into view the material 
con-tained in a note or even the entire other text — here all of Ulysses — to 
which that note refers. Scholarly articles situate themselves within a network of 
textual re-lations, most of which the print medium keeps out of sight and 
relatively difficult to follow — because the referenced (or linked) materials lie 
spatially distant from the reference mark. Electronic hypertext, in contrast, 
makes individual refer-ences easy to follow and the endre field of 
interconnections explicit and easy to navigate. Instant access to the whole 
network of textual references radically changes both the experience of 
reading and, ultimately, the nature of that which is read. If our putative Joyce 
article was linked, through hypertext, to all die other materials it cited, it would 
exist as part of a much larger system in which die to-tality might count more 
than die individual document; die article would now ap-pear WOVen more 
tightly into its context than would a print-technology counterpart. The case 
with which readers traverse such a system has further 
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consequences: for as they move through this web or network of texts, they 
con-tinually shift the center — and henee focus or organizing principie — of their 
in-vestigation and experience. Hypertext provides an infinitely re-centerable 
system whose provisional point of focus depends upon the choices made by a 
truly active reader. 

Hypertext thus presages a potential revolution in literary studies. However, 
an almost unlimited power to manipulate texts brings with it conceptual prob- 
lems of extreme difficulty which can be summed up by the questions: "What 
is a unit of text?" and "What are the relevant links between units?" Among 
tradi-tional textual units the best-recognized and most functional ones are the 
word, the sentence, and the book. To think of them as commensurable units 
on a linear scale of magnitude is natural, but misleading. A word is a 
conceptual unit, a sentence a syntactical one, a book a unit whose identity is 
largely determined by its traditional status as a physical object. Nonetheless, 
they are units that can be handled by many kinds of textual aids developed 
over the whole period of liter-acy. But between the sentence and the book (in 
terms of magnitude) we find such units as footnotes, paragraphs, chapters 
and essays; these are less amenable to definition, because they are largely 
informal means of organizing thought. 6 For the same reason, however, they 
are likely to be important elements for build-ing hypertext structures: they are 
the kind of mental "chunks" that we use to break a complex issue into 
components and make it intelligible. In addition, we are beginning to imagine 
new textual units, not yet codified or even named, that will be specific to the 
hypertext environment. They will come into being and pass away in the 
dynamic virtual text of the computer, producis of such broad cogni-tive 
principles as identity, association and structure. 7 

These deep theoretical implications of hypertext converge with some major 
points of contemporary literary and semiological theory, particularly with Der- 
rida's emphasis on decentering, with Barthes's conception of the readerly 
versus the writerly text, with post-modernism's rejection of sequential 
narratives and unitary perspectives, and with the issue of "intertextuality." In 
fact, hypertext creales an almost embarrassingly literal embodiment of such 
concepts.... 



Finally, hypertext can be expected to have important institutional as well as 
intellectual effects, for it is at the same time a form of electronic text, a 
radically new information technology, a mode of publication, and a resource 
for collabo-rative work. "Both an author's tool and a reader's medium, a 
hypertext document system allows authors or groups of authors to link 
information together, create paths through a Corpus of related material, 
annotate existing texts, and create notes that point readers to either 
bibliographic data or the body of the refer- 
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enced text.... Readers can browse through linked, cross-referenced, 
annotated texts in an orderly but nonsequential manner." 8 Such electronic 
linking shifts the boundaries between individual works as well as those 
between author and reader and between teacher and student. It also has 
radical effects upon our experience of author, text, and work, revealing that 
many of our most cherished, most com-monplace, ideas and altitudes 
towards literaiy production are the result of the particular technology of 
information and cultural memoiy that has provided the setting for them. This 
technology — that of the printing-press, the book, and the library — engenders 
certain notions of authorial property, authorial uniqueness, and a physically 
isolated text that hypertext makes untenable. Hypertext his-toricizes many of 
our most commonplace assumptions, forcing them to descend from the 
ethereality of abstraction and appear as corollary to a particular technology 
and historical era. We can be sure that a new era of computerized textu-ality 
has begun; but what it will be like we are just beginning to imagine. 

From Hypertext to Hypermedia 

Expository prose, with its linear and propositional structures, has been too 
much identified with the privileged form of reason itself. Hypertext provides a 
better model for the mind's ability to re-order the elements of experience by 
changing the links of association or determination between them. But 
hypertext, like the traditional text from which it derives, is still a radical 
reduction — to a schematic visual code — of what was originally a complex 
physical and intellectual experience, engaging all the five senses. Hypermedia 
takes us even closer to the complex interrelatedness of everyday 
consciousness; it extends hypertext by re-integrating our visual and auditory 
faculties into textual experience, linking graphic images, sound and video to 
verbal signs. Hypermedia seeks to approxi-mate the way our waking minds 
always make a synthesis of information received from all five senses. 
Integrating or (re-integrating) touch, taste and smell seems the inevitable 
consummation of the hypermedia concept. 



Consciousness itself is a continuous linking and re-structuring of images 
se-lected from past, present and future; from the real and the imaginary; from 
the internal and external realms of experience. Current hypermedia programs 
have taken only a few, faltering steps towards electronic representation of 
human memory, fantasy and cognition. Nonetheless, hypermedia is, in 
conception at least, a much better model of the mind's typical activities than 
exists in the se-verely restricted code of linear prose. We can argue, 
therefore, for a natural pro-gression from the printed word to hypertext and 
hypermedia — analogous to the 
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progression from painting to still photography, to silent movies, and now to 
movies with color and sound. This is not to claim that the newer media will al- 
together supersede the older ones. The black and white photograph remains 
viable, but is no longer the absolute standard of representation that it was in the 
nineteenth century. Similarly, the printed book will remain a central element of 
culture even as the new ways of interacting with texts make their own claims on 
our attention.... 



reconfiguring the text Dispersing the Traditional Text 

Although in some not-so-distant future all individual texts may electronically link 
to one another, thus creating metatextual structures of a kind only partly 
imaginable at present, less far-reaching forms of hypertextuality have already ap- 
peared. 9 We already have works composed in hypertext that join blocks of text by 
electronic links to each other and to such graphic supplements as illustrations, 
maps, diagrams, visual directories and overviews. Second, there are the metatexts 
formed by interlinking individual sections of individual works. A third case is the 
adaptation for hypertextual presentation of material conceived in book technol- 
ogy. Such adaptations can work with textual units already given by the author, 
such as the individual sections of In Memoriam. 10 Conversely, one may, in the 
manner of Barthes's treatment of "Sarrasine," impose one's own lexias upon a 
work not explicitly divided into sections. 

A fourth kind of hypertext puts a classical linear text, with its order and fix-ity, 
at the center of the structure. The composer then links various supplemen-tary 
texts to this center, including critical commentary, textual variants, and 
chronologically anterior and later texts. In this case, the original text, which re- 
tains its oid form, becomes an unchanging axis from which radiate linked texts 
that surround it, modifying the reader's experience of this original text-in-a-new- 
context. 11 

When compared to text as it exists in print technology, all these forms of 
hypertext evince varying combinations of atomization and dispersal. Unlike the 



spatial fixity of printed text, no one state of an electronic text is ever final; it can 
always be changed. Hypertext builds in a second fundamental mode of variation, 
since electronic links or reading pathways among individual blocks permit dif- 
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ferent paths through a text. The numerating rhetoric of "first, second, third" so 
ivell suited to linear text may appear within individual blocks of text but cannot 
control the unfolding of understanding in a medium that encourages readers 
to choose various paths, rather than following a fixed and linear one. 

From a literary perspective based on book technology, the effects of elec- 
tronic linking may appear harmful and dangerous. The notion of an individual, 
discrete work becomes increasingly undermined and untenable within this 
form of information technology, as it already has within much contemporary 
critical theory. Hypertext linking, reader control, and continual re-structuring 
not only militate against modes of argumentation to which we have become 
accustomed, but they have other, more general eflects. The reader is now 
faced by a kind of textual randomness. The writer, conversely, loses certain 
basic controls over his text: the text appears to break down, to fragment and 
atomize into constituent el-ements (the lexia or block of text), and these 
reading uriits take on a life of their own as they become more self-contained 
because they are less dependent on what comes before or after in a linear 
succession. 

At the same time that the individual hypertext block has looser, or less de- 
termining bonds to odier blocks from the same work (to use a terminology that 
now threatens to become obsolete), it also can bond freely with text created 
by other authors. In fact, it bonds with whatever text links to it, thereby 
dissolving notions of the intellectual separation of one text from others as 
some chemicals destroy the cell membrane of an organism. Destroying the cell 
membrane will kill the cell; but destroying our conventional notions of textual 
separation has no fatal consequences. However, it will reconfigure the text and 
our expectations of it. As an individual block loses its physical and intellectual 
separation from others when linked electronically to them, it also finds itself 
dispersed into them. The necessary contextualization and intertextuality 
produced by situating individual reading units within a network of easily 
navigable pathways weaves texts, in-cluding those by different authors and 



those in nonverbal media, tightly together. One efiect is to weaken and even 
destroy altogether any sense of textual unique-ness, for what is essential in 
any text appears intermingled with other texts. Such notions are hardly novel 
to contemporary literary theory, but here again hypertext creales an almost 
embarrassingly literal reification or actualization of a principle or quality that 
had seemed particularly abstract and difficult in its earlier statement. Since 
much of the appeal, even charra, of these theoretical insights lies in their 
difficulty and even preciousness, this more literal presentation promises to 
disturb theoreticians, in part, of course, because it disturbs status and power 
relations within their — our — field of expertise. . . . 
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hypertext and the author 

Collaborative Work Hypertext demands new modes of reading, writing, 
teach-ing, and learning. In so doing it creales new understanding of 
collaborative learn-ing and collaborative work. To most people, "collaboration" 
suggests two or more scientists, songwriters, or the like working side by side 
on the same en-deavor, continually conferring as they pursue a project in the 
same place at the same time. Landow has worked on an essay with a fellow 
scholar in this manner. One of us, he relates, would type a sentence, at which 
point the other would ap-prove, qualify, or rewrite it, and then we would 
proceed to the next sentence. But probably a far more common form of 
collaboration (and the one used in this in-troduction, whose authors live two 
thousand miles apart) is "versioning," in which one worker produces a draft 
that another person dien later edits by mod-ifying and adding. Both of these 
models require considerable ability to work productively with other people, 
and evidence suggests that many people lack this quality. According to those 
who have carried out experiments in collaborative work, a third form proves 
more common than the first two: the assembly-line or segmentation model of 
working together, in which individual workers divide up the overall task and 
work entirely independently. This last mode is the form that most people 
engaged in collaborative work choose. 12 

Networked hypertext systems like Intermedia offer a fourth model of 
collaborative work that combines aspects of the three just described. By 
emphasizing the cooperative interaction of blocks of text, networked hypertext 
makes all ad-ditions to a system simultaneously a matter of versioning and of 
the assembly-line mode. Once ensconced within a network of electronic 
links, a document no longer exists by itself. It always has an active relation to 
other documents in a way that a book or printed document never can. From 
this crucial shift in the way texts exist in relation to others derive two 



principles that determine this fourth form of collaboration: First, any document 
placed on a networked system that supports electronically linked materials 
potentially exists in collaboration with any and all other documents on that 
system; second, any document electronically linked to any other document 
collaborates with it. 

According to the American Heritage Dictionary, to collaborate can mean ei- 
ther "to work together, especially in a joint intellectual effort" or "to cooperate 
treasonably, as with an enemy occupying one's countiy." The combination of 
labor, political power, and aggressiveness that appears in this dictionary 
defini- 
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tion well indicates some of the problems that arise when one discusses 
collabo-rative work. On the one hand, the notion of collaboration embraces 
notions of working together with others, of forming a community of action. 
This meaning recognizes that we all exist within social groups, and must make 
our contributions to them. On the other hand, collaboration also invokes a deep 
suspicion of work-ing with others, something both aesthetically as well as 
emotionally engrained since the advent of romanticism, which exalts the idea 
of individual effort to such i degree that it often fails to recognize or even 
suppresses the fact that artists and writers work collaboratively with texts 
created by others. 

Most of our intellectual endeavors involve collaboration, but we do not always 
recognize it. The rules of our culture, particularly those that define intellectual 
property and authorship, do not encourage such recognition; further, informa- 
tion technology from Gutenberg to the present — the technology of the book — 
systematically hinders full recognition of collaborative authorship. Hypertext, 
however, foregrounds this element of collaboration that other technologies of 
cultural memory suppress. It changes our sense of authorship, authorial 
property, and creativity (or originality) by moving away from the constrictions 
of page-bound technology. In so doing, it promises to have an effect on 
cultural and intellectual disciplines as important as those produced by 
earlier shifts in the technology of cultural memory that followed the invention 
of writing and print-ing... , 13 



part IV 



Immersion 

ortonHeilig 

Fhe Cinema of the 
Future" (1955) 



Morton Heilig, Sensorama. 
Courtesy of Scott Fisher, 

Thus, individually and collectively, 
by thoroughly applying the 
methodology ofart, the cinema 
ofthe Mure will become the first 
art form to reveal the new 
scientific woHd to man \n the full 



sensual vividness and dynamic 
vitality ofhis consciousness. " 
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« Morton Heiiig, through a combination of ingenuity, determination, and shee 
stubbornness, was the first person to attempt to create what we now cali virtu A 
reality. In the 1950s it occurred to him that al I the sensory splendor of life couli 
be simulated with "reality machines." Heiiig was a Hollywood cinematographer 
and it was as an extension of cinema that he thought such a machine might bu 
achieved. With his inclination, albeit amateur, toward the ontological aspiration! of 
science, Heiiig proposed that an artist's expressive powers would be enhancet by a 
scientific understanding of the senses and perception. H is premise was simple but 
striking for its time: if an artist controlled the multisensory stimulatior of the 
audience, he could provide them with the illusion and sensation of firsfc person 
experience, of actually "being there." 

Inspired by short-lived curiosities such as Cinerama and 3-D movies, it 
occurred to Heiiig that a logical extension of cinema would be to immerse the 
audience jn a fabricated world that engaged all the senses. He believed that by 
expanding cinema to involve not only sight and sound but also taste, touch, and 
smell, the traditional fourth wall of film and theater would dissolve, transport-ing 
the audience into an inhabitable, virtual world; a kind of "experience theater." 

Unable to find support in Hollywood for his extraordinary ideas, Heiiig moved to 
Mexico City jn 1954, finding himself jn a fertile mix of artists, filmmakers, 
writers, and musicians. There he elaborated on the multidisciplinary concepts 
found in this remarkable essay, "The Cinema of the Future/Though not widely 
read, it served as the basis for two important inventions that Heiiig patented in 
the 1960s.The first was theTelesphere Mask.The second, a quirky, nickelodeon- 
style arcade machine Heiiig aptly dubbed Sensorama, catapulted viewers into 
multisensory excursions through the streets of Brooklyn, and offered other 
adventures in surrogate travel. While neither device became a popular success, 
they influenced a generation of engineers fascinated by Heilig's vision of 
inhabitable movies. » 



r andemoniuni reigns supreme in the film industry. Every studio is hastily 
converting to its own "revolutionary" system — Cinerama, Colorama, Panoramic 
Screen, Cinemascope, Three-D, and Stereophonic Sound. A dozen marquees 
in Time Square are luring customers into the realm of a "sensacional new 
experience." 

Everywhere we see the "initiated" holding pencils before the winked eyes of 
the "uninitiated" explaining the mysteries of 3-D. The crides are lining up pro 
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and con concluding their articles profoundly with "after all, it's the story that 
counts." Along with other filmgoers desiring orientation, I have been reading 
these articles and have sadly discovered that they reflect this confusion 
rather than illuminate it. It is apparent that the inability to cope with the 
problem stems from a refusal to adopt a wider frame of reference, arid from a 
meager under-standing of the place art has in life generally. 

All living things engage, on a higher or lower level, in a continuous cycle of 
orientation and action. For example, an animal on a mountain ledge hears a 
rum-bling sound and sees an avalancha of rocks descending on it. It cries with 
terror and makes a mighty leap to another ledge. Here in small is the 
essence of a process that in animais and man is so automatic — so rapid — as 
to seem one indivisible act. By careful introspection, however, men have been 
able to stop its »apid flow, bring it into the light of consciousness, and divide it 
into three basic aliases. The first, observation (the noise and image of the 
boulders in our exam-jle), is the reception of isolated impressions or facts. The 
second, integration, is he combining of these isolated facts with the inner 
needs of the life force into an motional unity that prompts and controls action 
(the animal's sensation of dan-;er and terror). The third, action (the leap to 
safety), is a change in the creature's ihysical relation to the world. 

With the forming of society, different men concentrated 

on one of these three 
bases, and by learning to cast the results of their labor 

into concrete forms (that 
ould be passed from man to man, and generation 
to generation) they created sci- 
ice, art, and industry. These three have the same 

methods and aims in the so- 
al body as the mind, heart, and muscle do in 
individual man. Their goals are 
ear. For science it is to bestow the maximum 
knowledge on humanity. For art 



is to digest this knowledge into the deeper realms of 

feeling, generating emo- 
>ns of beauty and love that will guide the crude 

energies of mankind to con- 
ructive actions. And for industry it is to act on the 

material world so as to 



ocure more living energy for mankind. The success 

with which each field can 
proach its goal depends on its understanding of 
method. Science has come the 
)sest because it has uncovered the individual's 

scientific thought processes 
d codified it into a clear and systematic method of 

experimentation. Con- 
ously applying this method, it makes more 
discoveries in one year than 
iviously were made in millenniums. Writing, 
international mail, and interna- 
rial conferences have long been efficient ways of 

distributing its findings to 
manity. 



Industry, within the last one hundred years, has also made great strides to-"d 
its goal because production geniuses like Ford have rationalized it to the last 
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degree. They have instigated assembly line, mass production techniques 
that pour out more food, machines, and fuels in one year than were produced 
in cen-turies. The problem of distributing its bulky goods has been solved 
theoretically and only awaits practical application. 

It is the middle field, art, which today is furthest from its goal. The world is 
woefully barren of peaceful, tolerant, humanitarian feelings and the art that 
should create them. And this is because, as yet, art has evolved no clear-cut 
methodology to make it as efficient as science and industry in creating its 
prod-uct. Art is now struggling feverishly to achieve this, and only in the light of 
this struggle and the laws it seeks to establish will we be able to understand 
the in-novations that prompted this ardele. 

The laws of art, like those of science and industry, lie hidden in the subcon- 
scious of man. When a primitive man desired to convey to another man the 
complete emotional texture of an experience that occurred to him he tried 
to reproduce, as closely as possible, the elements that generated his own 
emotions. His art was very simple, being limited to the means provided by his 
own body. He used his voice to growl like the bear that attacked him, pumped 
his arms and legs to show how he climbed a tree, and then he blew on his 
listener's face to make him feel the hot breath of the bear. If he were a good 
storyteller, he would arrange these ehects in more or less the same order they 
originally happened to him. Of course, his listener would feel everything more 
intensely if he, and not his friend, were attacked by the bear. But aside from 
being impossible, this is not advisable, for by listening to his friend's story, he 
can have all the excitement, learn all the lessons, without paying the price for 
them. 

With time language became more complete. A specific word-sound 
became associated with the impressions, objects, and feelings in man's 
experience. Words were useful in conveying the general structure of an event 
to the mind, but could rarely quicken the listener's pulse the way fresh and 
direct contact with the original sense elements could — that is unless spoken 
by a very skilled narrator. And even then not a thousand of his choicest words 
could convey the sensation of yel-low better than one glance at yellow, or high 
C better than listening for one sec-ond to high C. And so side by side with 



verbal language they evolved more direct forms of communication — painting, 
sculpture, song and dance. They found they could bore deeper into 
experience by concentrating all their powers of observa-tion on one of nature's 
aspects and mastering the limited materials necessary to its expression. 
Materials became more complex and techniques more refined as each art 
form sought to exploit the full range and delicacy of its own domain. The few 
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ines scratched on a rock developed into the full glory of painting. The 
singing roice evolved into symphonic music and the few words into the rich 
fabric of po-;try. For all the apparent variety of the art forms created, there is 
one thread unit-.ng all of them. And that is man, with his particular organs of 
perception and jction. For all their ingenuity, a race of blind men could never 
have evolved painting. Similarly, no matter how much they appreciated 
movement through their ;yes a race of limbless men could never have 
developed dancing. Thus art is like i bridge connecting what man can do to 
what he can perceive. 

What we commonly refer to as the "pure arts" are those whose materials 
are jo simple, so pliable, that one artist can master them sufficiently to 
express his nner feelings to perfection. The painter fashions color, the 
musician notes, the joet words. Each additional impression of their artistic 
form is like an electric :harge driving the spectator higher and higher to 
peaks of pure and intense feel-ng that he rarely experiences in his daily life. 
The simple materials of the pure rts are apprehensible through only one 
sense, but this sense is not a necessary ondition ofpurity. Precision and 
subtlety of form achieved through control is the lecisive factor. 
Desiring to convey the full richness of 
experience in more lifelike form, men 
ave combined the pure arts into forms 
known as the "combined" or "secondary 
rts," such as opera, ballet, and theatre. Their 
effects were fuller, more spectacu- 
ir, but rarely deeper. The essential factor of 
control was missing. Not only did 
le artist have to master visual, musical, 
choreographic, and verbal materials, 



ot only did he have to limit the scope of his 
imagination to the practical limita- 
ons ofa theatre and depend on the 



collaborations of dozens of singers, 

painters, 

ancers, musicians, and actors, but even 
after he had masterminded every detail 
id rehearsed the cast into perfect form he 
had absolutely no way of fixing his 
•cation so that it could remain exactly the 
same whenever and wherever played, 
his was an impasse the artist could never 
surmount and never did, until the ar- 
ral ofa strange newcomer on the scene — the 
machine. The machine with its ge- 
us for tireless repetition and infinite 
exactitude was an extension of the limbs 
id wul of man. It could be trusted to perform 
all his purely mechanical opera- 



ms, freeing his energies for more creative tasks. 

In the form of the printing press, lithograph, radio, phonograph, and now 
levision, the machine has rapidly solved the second part of art's age-old prob-n 
— distribution. Painting, poetry, music, drama, and ballet can now reach mil- 
>ns of people about the globe as they never could before. But the machine has 
ine more. It has entered, as it has done in industry and science, into the very 
here of artistic creation itself, providing the artist with a much wider palette of 
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sense material and enabling him to mold them with precision into an aesthetic 
unity as he never could before. And it is the invasion of such a relentlessly effi- 
cient and logical apparatus as the machine into the humane and heretofore 
ro-mantic field of art that not only suggests but necessitates a clear, efficient 
methodology of art. 

If at this point we sean back over our very brief history of art forms it becomes 
apparent that the first law of such an artistic methodology must be: "The 
nature of man's art is fundamentally rooted in his peculiar psychic apparatus 
and is lim-ited by the material means at his disposal." Logically, then a proper 
science of art should be devoted to the revelation of the laws of his psyche 
and the invention ofbetter means. 

Although very little of it was conscious or intentional, nothing demonstrates 
this research and invention more dramatically than the cinema. The sense was 
vision — the material, light. The still camera had been invented but it could do 
no more than a skilled painter could do with time. But when in 1888 the 
Lumiere Brothers set up a little box before their factory and cranked away at it 
as a group of workers left, they did something no human being could ever do 
before. They captured visual movement in a form that could exactly 
reproduce the moving image as often as desired. Only after countless 
millenniums of existence had man learned how to do what his visual 
mechanism can do with no effort at all. With time every part of this new 
machine, from the lens to the film stock, was im-proved. Lenses were made 
faster and given wider angles. The iris became ad-justable and the film fmer- 
grained and faster. Always the criteria of invention were to reproduce as 
closely as possible man's miraculous mechanism of vision. The addition of 
color was inevitable. Man sees color, so must his mechanical eye. Now we 
have the so-called "revolutionary" 3-D and Wide Screen. The excite-ment 



and confusion are great but they need not be. First, 3-D was invented ovei 50 
years ago and shown at the Paris Exposition. Financial, not technical 
considerations held it up until 1953. The really exciting thing is that thest 
new devices have clearly and dramatically revealed to everyone what paint- 
ing, photography, and cinema have been semiconsciously trying to do all 
along — portray in its full glory the visual world of man as perceived by the human 
eye. 

Side by side with the invention of means to freeze visual movement, 
machines were developed that could (this also for the first time in human 
historv freeze sound. But again, the public's deep and natural urge for more 
complete re alism in its art had to wait on the wheel of finance until 1933. It is 
the additioi of sound that represents the really great "revolution" in the history of 
cinema. Foi 



"The Cinema oFtheFufure" 245 

with the addition of sound, cinema stepped irrevocably out of the domain of 
the "pure arts" into the camp of the "combined arts." Rather than attempting to 
por-tray the whole through the part, it now began attempting to portray the 
whole di-rectly. But with this tremendous difference from all other composite 
arts — it could do it without losing control or permanence. With the help of the 
machine two radically different sense materials, light and sound, could be 
dynamically combined into one work without losing any of the control, subtlety, 
or concrete-ness formerly attained only by the pure arts. Cinema was no 
longer just a visual art (notwithstanding the effort of some directors to keep it 
such by shooting visual films and pouring the sound track over it like some 
pleasant, superfluous goo), but had set itself the task of expressing in all its 
variety and vitality the full consciousness of man. 

Instead of continuing to stumble along this road with the system of hit and 
miss, let us, according to our first law, deliberately turn to life and study the na- 
ture of man's consciousness. 

Man's nervous system — sensory nerves, brain, and motor nerves — is the 
seat of his consciousness. The substance or componen! parts of this 
consciousness can be determined by the process of elimination. If a man lies 
still, or, due to some disease or drug, has his motor nerves blocked, his 
consciousness or wide-awakeness is not diminished in any way. If, however, 
he closes his eyes, it is. If he stops his ears, it is diminished further. If he 
pinches his nose and does not taste anything and avoids tactile impressions, 
his awakeness is diminished consider-ably. And if, as is done in anesthesia, 
all sensory nerves leading to the brain are blocked, he would lose 



consciousness completely. (Dreams and internal voices merely being sense 
impressions of former experiences stored away and served up iater by 
memory.) Thus we can state our second law: "Consciousness is a composite 
of all the sense impressions conveyed to the brain by the sensory part of the 
nervous system which can be divided into the great receiving organs — the 
eyes, ears, nose, mouth, and skin." 

By concentrating on one organ at a time, we can list the various elements 
af-fecting it. These are, for the eye, peripheral imagery — 180° horizontal X 150° 
vertical, three dimensionality, color and movement; for the ear, pitch, 
volume, rhythm, sounds, words, and music; for the nose and mouth, odors 
and flavors; and for the skin, temperature, texture, and pressure. These 
divisions — although purely subjective and dependent on vocabulary and 
techniques of reproduc-tion — are nonetheless useful for analysis. 

These elements are the building bricks, which when united create the 
sensual Porra of man's consciousness, and the science of art must devote 
itself to invent- 
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with the addition of sound, cinema stepped irrevocably out of the domain of 
the "pure arts" into the camp of the "combined arts." Rather than attempting to 
portray the whole through the part, it now began attempting to portray the 
whole di-rectly. But with this tremendous difference from all other composite 
arts — it could do it without losing control or permanence. With the help of the 
machine two radically difFerent sense materials, light and sound, could be 
dynamically combined into one work without losing any of the control, subtlety, 
or concrete-ness formerly attained only by the pure arts. Cinema was no 
longer just a visual art (notwithstanding the effort of some directors to keep it 
such by shooting visual films and pouring the sound track over it like some 
pleasant, superfluous goo), but had set itself the task of expressing in all its 
variety and vitality the full consciousness of man. 

Instead of continuing to stumble along this road with the system of hit and 
miss, let us, according to our first law, deliberately turn to life and study the na- 
ture of man's consciousness. 

Man's nervous system — sensory nerves, brain, and motor nerves — is the 
seat of his consciousness. The substance or component parts of this 
consciousness can be determined by the process of elimination. If a man lies 
still, or, due to some disease or drug, has his motor nerves blocked, his 
consciousness or wide-jwakeness is not diminished in any way. If, however, he 
closes his eyes, it is. If he stops his ears, it is diminished further. If he pinches 
his nose and does not taste inything and avoids tactile impressions, his 



awakeness is diminished consider-ably. And if, as is done in anesthesia, all 
sensory nerves leading to the brain are blocked, he would lose consciousness 
completely. (Dreams and internal voices merely being sense impressions of 
former experiences stored away and served up later by memory.) Thus we can 
state our second law: "Consciousness is a composite of all the sense 
impressions conveyed to the brain by the sensory part of the nervous system 
which can be divided into the great receiving organs — the eyes, ears, nose, 
mouth, and skin." 

By concentrating on one organ at a time, we can list the various elements 
af-fecting it. These are, for the eye, peripheral imagery — 180° horizontal X 150° 
vertical, three dimensionality, color and movement; for the ear, pitch, 
volume, A hythm, sounds, words, and music; for the nose and mouth, odors 
and flavors; md for the skin, temperature, texture, and pressure. These 
divisions — although aurely subjective and dependent on vocabulary and 
techniques of reproduc-ion — are nonetheless useful for analysis. 

These elements are the building bricks, which when united create the 
sensual brm of man's consciousness, and the science of art must devote itself 
to invent- 
ing techniques for recording and projecting them in their entirety. Celluloid film 
is a very crude and primitive means of recording light and is already being re- 
placed by a combination television camera and magnetic tape recorder. 
Similarly sound recording on film or plastic records is being replaced by tape 
recording. 

Odors will be reduced to basic qualities the way color is into primary colors, The 
intensity of these will be recorded on magnetic tape, which in turn will control 
the release from vials into the theatre's air conditioning system. In time all o] 
the above elements will be recorded, mixed, and prqjected electronically — a 
red of the cinema of the future being a roll of magnetic tape with a separate 
track foi each sense material. With these problems solved it is easy to imagine 
the cinema of the future. 

Open your eyes, listen, smell, and feel — sense the world in all its magnificeni 
colors, depth, sounds, odors, and textures — this is the cinema of the future! 
The screen will not fill only 5% of your visual field as the local movie screei 
does, or the mere 7.5% of Wide Screen, or 18% of the "miracle mirror" screei 
of Cinemascope, or the 25% of Cinerama — but 100%. The screen will curve 
pasi the spectator's ears on both sides and beyond his sphere of vision 
above anc below. In all the praise about the marvels of "peripheral vision," no 
one pausec to state that the human eye has a vertical span of 150° 1 as well as a 
horizontal on< of 180°. The vertical field is difficult, but by no means 
impossible, to provide Planetariums have vertical peripheral vision and the 
cinema of the future wil provide it along similar lines as shown in the 
accompanying drawing. This 180' X 150° oval will be filled with true and not 
illusory depth. Why? Because a\ demonstrated above this is another 
essential element of man's consciousness Glasses, however, will not be 
necessary. Electronic and optical means will be de-vised to create illusory 
depth without them. 



Cinemascope, despite all the raving of its publicity men that it is the "crown-ing 
glory" of motion picture development, represents one small step forward, anc 
one big one backward. Its increase of screen image from 5 to 18% of man's 
visua field is a definite improvement although there is still 82% to go. It has, 
howevei regressed substantially in clarity. One reason that few critics noticed 
for Citt erama's excellent illusion of reality is its extraordinary clarity. The 
human eye ij one of the most perfect in the animal kingdom. It is not spotty, 
out-of-focus. of jumpy the way average movie images are. The image it 
records is limpid, razor sharp, and solid as a rock, and Cinerama, by using 
three film strips instead of one and specially designed projectors, makes a 
great advancement towards this per fection. Cinemascope, on the other 
hand, by still using only one film strip u cover two and one-half the normal 
screen area, is also magnifying grain, and soft 
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ling the focus two and one-half times, making clarity much worse than it is on 
ie normal screen. 2 The electrically created image of tomorrow's film will be 
srfect in focus and stability — the grain and spots vanishing along with the film 
ock. 

Stereophonic sound will be developed so that the spectator will be enclosed 
ithin a sphere, the walls of which will be saturated with dozens of speakers, 
ounds will come from every direction — the sides, top, back and bottom — as 
iey do in real life. 

The large number of speakers will permit a much better identity of image and 
)und than is achieved now where the sound leaping from one distant 
speaker > another is either behind or ahead of the image. The air will be 
filled with dors and up to the point of discretion or aesthetic function we will 
feel changes F temperature and the texture of things. We will feel physically 
and mentally ansported into a new world. 

Yes, the cinema of the future will far surpass the "Feelies" of Aldous Huxley's 
rave New World. And like many other things in this book that are nightmarish 
ecause superficially understood, it will be a great new power, surpassing 



con-;ntional art forms like a Rocket Ship outspeeds the horse and whose 
ability to estroy or build men's souls will depend purely on the people behind 
it. 

The mastery of so many sense materials pose another problem — selection, 
sople already complain about the excess of realism in films and say the new in- 
jntions shall plunge us from bad to worse. Although the spirit of their complaint 
valid, their use of the word "realism" is not. "Realism," or, in aesthetic terms, 
jxperience," is that something which is created by the unity of the outer world 
ith the inner. No matter how extensive the artist's means, he must use them to 
•ovoke more of the spectator's participation, not less. For without the active 
par-:ipation ofthe spectator there can be no transfer of consciousness, no art. 
Thus t is never "too" realistic. When either too much or too little is given, there 
just n't any "realism." Poor use of cinema's remarkable new powers is no more 
of a se against them than daubing with oils is a case against painting. 

It is estimated that each sense monopolizes man's attention in the following 
'oportions: 



Sight 


70% 


Hearing 


20% 


Smell 


5% 


Touch 


4% 


Taste 


1% 
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Men can have their attention led for them as a bird will do by flying across < 
empty sky, or can willfully direct it as everyone does at the dinner table when si 
gling one voice out of the maze of chatter. In each case the criterion is "what 
the point of greatest interest and significance to me?" Thousands of sense ii 
pressions stimulate the sensory nerves every second of the day, but only one 
a few are permitted to enter the realm ofhigher consciousness at a time. Ti 
organ that screens them out is the brain. The brain is the storehouse — the mei 
ory of the physical and spiritual needs of the individual, and through him of t 
human race, and it is according to this criteria — "what is beneficial for the d 
velopment of the individual and racial life force?" that a decision is made. We c 
now state the third law of our methodology of art: "The brain of man shi 
rapidly from element to element within each sense and from sense to sense in t 
approximate proportion of sight, 70%; hearing, 20%; smell, 5%; touch, 4%; ai 
taste, 1%, selecting one impression at a time according to the needs of indivi 
ual and racial development." These unite into the dynamic stream of sensatio 
we cali "consciousness." The cinema of the future will be the first direct, ca 



piete and conscious application of this law. Since the conventional movie sere 
filis only 5% of the spectator's field of vision, it automatically represents his pa of 
visual attention and the director needs only to point his camera to control I 
point of attention. But with the invention ofmeans to ful 100% of the spectato 
field of vision with sharp imagery, he must solve the problem of visual attenri 
another way or lose his main aesthetic power. 

Every capable artist has been able to draw men into the realm of a new t 
perience by making (either consciously or subconsciously) a profound studj 
the way their attention shifts. Like a magician he learns to lead man's attend 
with a line, a color, a gesture, or a sound. Many are the devices to control the sp 
tator's attention at the opera, ballet, and theatre. But the inability to eliminate I 
unessential is what loosens their electrifying grip on the attention of a specta 
and causes them to remain secondary arts. 

The evolution of the aesthetic form of cinema can, in a way, be described a 
continuation of the artist's struggle to master attention. Griffith began using i 
"close-up" to draw the spectator's attention to a significan! visual detail. Leu 
with narrow focus fields were devised to throw foreground and background i of 
focus, riveting the eye only on the sharp part of the image. Pudovkin develoj the 
close-up in time by varying camera speeds to parallel die varying intensitj 
man's observations. Eisenstein proclaimed "montage" and Griffith discovei 
"parallel cutting," both magnificent weapons in the director's arsenal of attena 
Shots and scenes could now be shifted with the same freedom and rapidity p 
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sible in man's natural observation or imagination. Sound arrived with undimin- 
ished intensity, but in time it too became refined in conteni, pitch, and volume, 
sometimes dominating the scene, sometimes leaving it completely, leading the 
ear as precisely as the eye. But like the search for an additional number of 
sense ma-terials, the principle involved in this refinement of attention were 
mostly stum-bled on by accident — rarely searched for deliberately, and never 
formulated consciously. 

Again, the only place to search is in the mind of man. We must try to learn 
how man shifts his attention normally in any situation. 

Suppose we are standing on a hilltop overlooking the countryside. First we 
are struck by the huge sweep of the view before us. Then we notice the vivid 
green of the fields and the sunshine. Then the silent expansion and rolling of 
a cumulus cloud entrances us. We feel a warm gust of wind and our nostrils 
dilate at the smell of new-mown hay. Suddenly, our ears sharpen as the shriek 
of ajet plane cuts the air. We cannot see it, but we linger on the way its high- 
tone low-ers in pitch and fades away. Here is an example of how attention 



shifts from one element (space — color — then movement, in our example) 
within a single sense (the eye) and from one sense to another (the eye, the 
skin, the nose, the ear). In each moment it fixes itself, if for only an instant, on 
one sense element to the par-tial or complete exclusion of all others. 

In life, only the object being observed is in focus. The area of focus is not 
hec-essarily rectangular, including everything in the same plane, as it is in 
today's films, but can be circular, triangular, vertical, or horizontal, depending 
on the shape of the objects of interest. Electrical and optical means will be 
developed to duplicate this flexibility, retaining the hazy frame of peripheral 
vision as the human eye does for added realism. Naturally, the great visual 
oval of the camera field will include, exclude, move closer, and recede as it 
does in life. This zone of focus will generally be at the center of the visual field, 
but it will be free to shift up and down, or around to the sides, leading the eye 
wherever it goes. The di-rection, quality, and intensity of all other sense 
elements will be controlled and pin-pointed in the same subtle manner. 

Each basic sense will dominate the scene in roughly the same proportion 
we found them to have in man. That is, sight, 70%; sound, 20%; smell, 5%; 
touch, 4%; and taste, 1%. Nature turns them on and offwithout a whimper 
but film-makers once in possession of a new power usually cling to it like a 
drowning man to a life raft. Eye irritating colors, ear deafening dialogue, and 
soul sicken-ing music are loaded one on top of another just to "make sure 
the point gets across." The cinema of the future will turn any and all of it off, 
including the vi- 



250 MortonHeilig 

sual part, when the theme calis for it. For, and it cannot be stressed too 
strongly, the cinema of the future will no longer be a "visual art," but an "art 
of con-sciousness." 

When a great many sense materials are presented in sharp focus 
simultane-ously the spectator must do his own selecting. He is no longer being 
led along by a work of art, but must begin with great fatigue to create his own 
patterns. This situation is so life-like that it gives the spectator the sensation of 
being physically in the scene. For example, in Cinerama's famous roller 
coaster sequence, the spectator's body, not his soul, is riding the roller 
coaster. This is a tremendous faculty and will, I am sure, be used to great 
effect in the cinema of the future, but it must be used with great discretion. For 
aside from being very tiring, after one too many loops, the spectator may be 
so thoroughly convinced that he is shoot-ing the chutes as to throw up on the 
lady in front of him. As stated before, art is a specific technique for living 
vicariously, of weeping without actually losing a loved one, of thrilling to the 
hunt without being mangled by a lion, in short of reaping the lessons and 



spiritual nourishment of experience without any loss. The solution of the 
problem of focus will invalidate the opinion that the wide screen is no good 
for "the intimate thing." If man can have intimate moments in life with his 
peripheral vision, stereophonic hearing, smell, and touch, so can his art. 

It would seem from the preceding analysis that my conception of the function 
of the cinema of the future is to faithfully reproduce man's superficial and im- 
mediate perception of the world about him. Nothing could be further from the 
truth. The history of art demonstrates over and over again that some of the 
most valid experiences come from the inner and not the outer world. But the 
history, not only of art but any other human endeavor, also proves that the 
outer precedes the inner. The outer world supplies the raw materials of 
creation. Man cannot originate. He can only take the forces of nature and 
rearrange them into shapes more friendly to his own existence. Just as nature 
had to provide water, iron, fire, and the laws of thermodynamics before 
someone could invent the steam engine, so nature must supply man with raw 
impressions before he can fashion them into an imagery more meaningful 
and useful to himself. The first task of painting was to copy the world, and only 
when the camera relieved it of this mirror-like function was it really free to 
explore the full range of man's fantasy. At first, mo-tion picture cameras and 
sound recorders could not even capture the simplest as-pects of man's 
perception of the outer world. Now, though still far from matching some of 
these, they are far superior to others. Slow motion, fast motion, and in-frared 
ray photography are able to "see" things no human eye can. 
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Supersensitive microphones are now able to "hear" sounds way beyond the 
mge of human ears. Similarly, directors at first had to be conteni with what the 
•atural scene about them oflered. Then, in studios, they began to select and 
irrange what went before the lens. By building sets, and developing trick 
phonography, they could set the world of history and fantasy before the lens. 
Then, by perfecting the technique of animation, they could do without bulky 
sets and intricate models entirely and give free reign to their wildest 
imagination. 

Sound has followed a similar evolution — from the objective to the subjective 
world. First we recorded only natural sounds, or the sounds created by 
human voices and musical instruments. Then we invented a whole series of 
odd new sound-making instruments and set them about the microphone. 
Now people like Norman McLaren are dispensing with expensive 
instruments and microphones entirely and are creating sound never heard 
before by painting directly on the sound track. 

These developments bring us to our fourth law: "In his creative process, 
man is imposed on by outer impressions. He learns the secrets of their 



basic principles through imitation and then subjects these to the needs of his 
own ex-pression. He goes from reception to imitation to creation, i.e., from 
portraying the outer to portraying the inner world." 

This law will inevitably hold true for the cinema of the future. While it still 
must learn to faithfully reproduce man's outer world as perceived in his con- 
sciousness, it will eventually learn to create totally new sense malcriais for 
eachof the senses — shapes, movements, colors, sounds, smells, and tasles — 
ihey havenever known before, and lo arrange ihem inlo forms of 
consciousness never before experienced by man in his conlacl wilh ihe outer 
world 

The thealre will provide for ihe full exercise of ihe social instincts. It will 
incorporate a promenade and cafe around ihe ihealre proper. The film will nol 
be presenled as "enterlainmenl" but as an evening of community culture. A 
speaker will review ihe personalilies in and hislory of ihe film being viewed. Afler 
ihe performance ihe audience will crilicize ihe film in a discussion facililated by 
television relays and led by a moderator. The audience will be able to 
continue the discussion in the cafe-lounge or on the promenade where they 
can see, be seen, and enjoy the evening in a thoroughly social fashion. Thus, 
individually and col-lectively, by thoroughly applying the methodology of art, the 
cinema of the future will become the firsl arl form to reveal the new scientific 
world lo man in ihe full sensual vividness and dynamic vilality of his 
consciousness. 

— translated by uri feldman 
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Ivan Butherland 

"The Ultimate Gisplay" (1965) 
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han Sutherland, Head- 
Mounted Display (worn by Don 
Vickers). Courtesy ofilniversity 
ofiltah. 



"The ultimate display would, ofcourse, be a room within which the 
computer can control the existence of matter. A\ chair displayed in 
such a room would be good enough to si in. Handcuffs displayed \n 
such a room would be confining t and a bullet displayed in such a 
room would be fatal. With appropriate programming such a display 
could literally be the Wonderlandinto which Alice walked. " 
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« I van Sutherland made his first significant contribution to computer science 
jn 1963 as a twenty-two-year-old graduate student at Mil, where he pioneered 
the field of Interactive computer graphics with his legendary Sketchpad system. 
Later, in 1965, while conducting his initial research jn immersive technologies, 
Sutherland wrote "The Ultimate Display," in which he made the first advance to- 
ward marrying the computer to the design, construction, navigation, and habi- 
tation of virtual worlds. This essay has since become known by generations of 
scientists, computer artists, and media theorists as the seminal statement to 
define the field of computer-based virtual reality. 

Morton Heilig provided the first glimpse of a simulated reality, but had no 
means to pursue his vision. Sutherland, on the other hand, had access to the abun- 
dant resources of ARPA. He predicted that advances in computer science would 
eventually make it possible to engineer virtual experiences that were convincing 
to the senses. Sutherland understood that pure information can be expressed in 
any number of forms, through an interface of the engineer's choosing. That in- 
terface might be the unes of type on a screen, or, he reasoned, jt could be a fully 
realized three-dimensional environment. While this speculation seems influenced 
more by science fiction than scientific fact, Sutherland believed jn the ineffable 
potential of computers to transform the abstract nature of mathematical con- 
structions into habitable, expressive worlds jn the spirit of Lewis Carrol I's Alice 
in Wonderland, 

In this essay, Sutherland presents a survey of the hardware and software that 
would someday constitute the computer-driven "reality" engine. Although it was 
several years before the invention of the personal computer, Sutherland had al- 
ready conceived an Interactive, immersive system that went well beyond the key- 
board, light pen, and joystick, which were the cutting-edge technologies of the 
day. In 1966, Sutherland took a crucial step toward the implementation of his 



vision by inventing the head-mounted display — a helmet-shaped apparatus de- 
signed to immerse the viewer jn a visually simulated 3-D environment. He com- 
pleted the prototype jn 1970 at the University of Utah, and fondly nicknamed it 
the "Sword of Damocles," because the mass of hardware threatened to 
decapitate the immersant.This invention led to the development of the more 
sophisti-cated and visually arresting representations of reality that became 
possible with advances in computing power and 3-D graphics. » 
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live in a physical world whose properties we have come to know well 
through long familiarity. We sense an involvement with this physical world which 
gives us the ability to predict its properties well. For example, we can predict 
where objects will fall, how well-known shapes look from other angles, and how 
much force is required to push objects against friction. We lack 
corresponding familiarity with the forces on charged particles, forces in 
nonuniform fields, the effects of nonprojective geometric transformations, and 
high-inertia, low friction motion. A display connected to a digital computer 
gives us a chance to gain familiarity with concepts not realizable in the 
physical world. It is a looking glass into a mathematical wonderland. 

Computer displays today cover a variety of capabilities. Some have only the 
fundamental ability to plot dots. Displays being sold now generally have built in 
line-drawing capability. An ability to draw simple curves would be useful. Some 
available displays are able to plot very short line segments in arbitrary direc- 
tions, to form characters or more complex curves. Each of these abilities has a 
his-tory and a known utility. 

It is equally possible for a computer to construct a picture made up of 
colored areas. Knowlton's movie language, BEFLIX, 1 is an excellent example 
of how computers can produce area-filling pictures. No display available 
commercially today has the ability to present such area-filling pictures for 
direct human use. It is likely that new display equipment will have area-filling 
capability. We have much to learn about how to make good use of this new 
ability. 

The most common direct computer input today is the typewriter keyboard. 
Typewriters are inexpensive, reliable, and produce easily transmitted signais. 



As more and more on-line systems are used, it is likely that many more 
typewriter consoles will come into use. Tomorrow's computer [user] will 
interact with a computer through a typewriter. He ought to know how to 
touch type. 

A variety of other manual-input devices are possible. The light pen or RAND 
Tablet stylus serve a very useful function in pointing to displayed Items and in 
drawing or printing for input to the computer. The possibilities for very smooth 
interaction with the computer through these devices is onlyjust beginning to 
be exploited. RAND Corporation has in operation today a debugging tool 
which recognizes printed changes of register contents, and simple pointing 
and moving motions for format relocation. Using RAND's techniques you can 
change a digit printed on the screen by merely writing what you want on top of 
it. If you want to move the contents of one displayed register into another, 
merely point to the 



"The Ultimate Uisplay" 255 

frrst and "drag" it over to the second. The facility with which such an interaction 
system lets its user interact with the computer is remarkable. 

Knobs and joysticks of various kinds serve a useful function in adjusting pa- 
rameters of some computation going on. For example, adjustment of the 
viewing angle of a perspective view is conveniently handled through a three- 
rotationjoy-stick. Push-buttons with lights are often useful. Syllable voice input 
should not be ignored. 

In many cases the computer program needs to know which part of a 
picture the man is pointing at. The two-dimensional nature of pictures makes 
it impos-sible to order the parts of a picture by neighborhood. Converting 
from display coordinates to find the object pointed at is, therefore, a time- 
consuming process. A light pen can interrupt at the time that the display 
circuits transfer the item being pointed at, thus automatically indicating its 
address and coordinates. Spe-cial circuits on the RAND Tablet or other 
position input device can make it serve the same function. 

What the program actually needs to know is where in memory is the struc- 
ture which the man is pointing to. In a display with its own memory, a light pen 
return tells where in the display file the thing pointed to is, but not necessarily 
where in main memory. Worse yet, the program really needs to know which 
sub part of which part the man is pointing to. No existing display equipment 
computes the depths of recursions that are needed. New displays with analog 
mem-ories may well loose the pointing ability altogether. 



other types of display 

If the task of the display is to serve as a looking-glass into the mathematical 
won-derland constructed in computer memory, it should serve as many senses 
as pos-sible. So far as I know, no one seriously proposes computer displays of 
smell, or taste. Excellent audio displays exist, but unfortunately we have little 
ability to have the computer produce meaningful sounds. I want to describe 
for you a kines-thetic display. 
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The force required to move ajoystick could be computer controlledjust as 
the actuation force on the controls of a Link Trainer are changed to give the 
feel of a real airplane. With such a display, a computer model of particles in an 
elec-tric field could combine manual control of the position, of a moving 
charge, replete with the sensation of forces on the charge, with visual 
presentation of the 

charge's position. Quite complicated "joysticks" with force feedback capability 
exist. For example, the controls on the General Electric "handyman" are 
noth-ing but joysticks with nearly as many degrees of freedom as the human 
arm. By use of such an input/output device, we can add a force display to our 
sight and sound capability. 

The computer can easily sense the positions of almost any of our body 
mus-cles. So far only the muscles of the hands and arms have been used for 
computer control. There is no reason why these should be the only ones, 
although our dex-terity with them is so high that they are a natural choice. Our 
eye dexterity is very high also. Machines to sense and interpret eye motion 
data can and will be built. It remains to be seen if we can use a language of 
glances to control a computer. An interesting experiment will be to make the 
display presentation depend on where we look. 

For instance, imagine a triangle so built that whichever corner of it you look 
at becomes rounded. What would such a triangle look like? Such 
experiments will lead not only to new methods of controlling machines but also 
to interesting understandings of the mechanisms of vision. 

There is no reason why the objects displayed by a computer have to 
follow the ordinary rules of physical reality with which we are familiar. The 
kinesthetic display might be used to simulate the motions of a negative mass. 



The user of one of today's visual displays can easily make solid objects 
transparent — he can "see through matter!" Concepts which never before had 
any visual representation can be shown, for example the "constraints" in 
Sketchpad. 2 By working with such displays of mathematical phenomena we 
can learn to know them as well as we know our own natural world. Such 
knowledge is the major promise of computer displays. 

The ultimate display would, of course, be a room within which the computer 
can control the existence of matter. A chair displayed in such a room would be 
good enough to sit in. Handcuffs displayed in such a room would be 
confming, and a bullet displayed in such a room would be fatal. With 
appropriate program-ming such a display could literally be the Wonderland into 
which Alice walked, acott Fisher 



Virtual InterFace 
Environments" (1989) 
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Virtual Interface Environmental Workstation. Photo by W. Fisher and 
S. Fisher. Courtesy ofScott Fisher, NASA-Ames Research Center. 

'The possibilities of virtual realities, jt appears, are as Hmitless as 
the possibilities of reality. They can provide a human interface that 
disappears — a doorway to other worlds. " 
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« Scott Fisher is one of the new breed of artist-engineet-s who have emerge in 
recent years from interdisciplinary academic institutions such as MIT's Ced ter 
for Advanced Visual Study. While a student jn the 1970s, Fisher worked o the 
groundbreaking Aspen Movie Map project, jn which the viewer-participal 
navigates through an interactive video-disc representation of Aspen, Colorad! 
This form of environmental simulation, dubbed "surrogate travel," was a stron 
influence on Fisher's subsequent work, which integrates scientific research jn vi 
tual reality with the artistic application of immersive environments. Inspired t 
experimenta in simulation such as the nineteenth-century zoetrope, the niel 
elodeon, Cinerama, Morton Heilig's Sensorama, and Ivan Sutherland's heaj 
mounted display, Fisher's work, which combines these influences, defines oj 
contemporary notion of virtual reality. 

This article documents Fisher's seminal research conducted in the late 1981 at 
the NASA-Ames Research Center in Mountain View, California, where I worked 
on the Virtual Environment Workstation (VIEW) project. Intending 1 extend the 
possibilities of the virtual environment beyond the "surrogate travel of the Aspen 
project, Fisher set out to develop an interface that would enga< all the senses, 
thrusting the viewer into a realm of full sensory jmmersion.TI NASA system 
included an updated version of the head-mounted display, wi stereoscopic 
images that provided depth of field, a major advancement over ti monoscopic 
vision of Sutherland's earlier device. Fisher added headphones f surround-sound, 
a microphone for speech recognition, and, in collaboration w scientistTom 
Zimmerman, adapted the "dataglove" — the wired glove worn the user that 
makes jt possible to grasp virtual objects in cyberspace. 



This multisensory interaction with cybernetic devices created the powerfu 
lusion of entering a digitized landscape. By pursuing Heilig's concept of exp< 
ence theater, Fisher made a significant advance toward what he terrj 
"telepresence" — the projection of the self into a remote location or viri 
world.This experience couid range from simulations of undersea exploratiod 
flights through abstract, imaginary space. Influenced by the work of artist My 
Krueger, Fisher's research into the use of collaborative telepresent environrr* has 
led to such diverse applications as telesurgery, architectural walkthrou A on-line 
communities, and multi-user games. » 
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Un the morning ofjuly 29, 2001, Helen and Graham Lesh commuted to work in 
their accustomed manner. They got up from the breakfast table, kissed each 
othergoodbyefor the morning, and walked down the hall to their respective work- 
spaces, where they slipped into their datasuits and donned their virtual 
environment headsets. 

An outside observer would see Helen standing almost immobile in the center of 
her dimly lit space, making a series offine movements with her hands. An outside 
observer would not be seeing the same environment that Helen saw, however. 
Helen, a plastic microsurgeon, found herself, as expected, infidl sensory contact 
with her virtual operating theater: The bright white lights, the blue-garbed OR 
crew, and the phalanxes ofequipment that surrounded them, looked and sounded 
exactly as they would in a real operation. Reachingfor a scalpel, murmuring com- 
mands to her interface, Helen preparedfor an hour ofintensive rehearsalfor the 
operation that would take place in the real, life-and-death operating theater at 
University Hospital thefollowing day. 

While Helen was zooming her view in on facial capillaries modeled on those of 
tomorrow's patient and etching virtual tissue afew hundred molecules at a time 
with a micromanipulated laser scalpel, Graham was in his workspace, trying to 
create a new pharmaceutical. An outside observer would think that Graham was 
mirmng a wrestling match with himself, or recapitulating his infancy, for Graham 
was on allfours, his head aimed at a point in space near one corner ofthe 
workspace. In Graham 's reality, he was crawling around a complex molecule, 
reaching for other molecules and trying tofit them together, like a kind ofdynamic 
Tinker-toy. 



For the time being, Graham and Helen are fictitious, but they are using 
perfectly plausible technological descendants ofthe virtual environment 
display systems that have been developed over the past four years at 
NASA's Ames Research Center in the Aerospace Human Factors Research 
Division [Fisher et al., 1986, 1987,1988].' 
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Here, in faci, there is a graphic simulation of a hemoglobin molecule that a 
person can fly through and move around in physically, as if reduced to 
molecular size. A head-mounted, wide-angle, stereoscopic display system 
controlled by operator position, voice, and gesture has been developed for 
use as a multipur-pose, multimodal interface environment. 1 The system 
provides a multisensory, Interactive display environment in which a user can 
virtually explore a 360-degree synthesized or remotely sensed environment 
and can viscerally interact 
^bu bcoTThsner 

with its components. Our objective has been to develop a new kind of 
interface that would be very closely matched to human sensory and cognitive 
capabilities. The effect of immersing one's sensoria in even the crudest 
prototypes of these interfaces has led many who have experienced it to refer 
to the experience as a kind of virtual reality. We prefer to use the term virtual 
environment to empha-size the ability to completely immerse a subject in a 
simulated space with its at-tendant realities. 



A bit of history: the evolution of personal simulation and telepresence 
environments 

Matching visual display technology as closely as possible to human cognitive 
and sensory capabilities in order to better represent "real" experience has 
been a major ambicion for research and industry for decades. One example is 
the de-velopment of stereoscopic movies in the early 1950s, in which a 
perception of depth was created by presenting a slighdy different image to 
each eye of the viewer. In competition with that during the same era was 
Cinerama, which in-volved three different projectors presenting a wide field of 



view display to the au-dience. Because the size of the projected image was 
extended, the viewer's peripheral field of view was also engaged. More 
recently, along the same lines, the Omnimax projection system situales the 
audience under a huge hemispherical dome, and the film image is 
predistorted and projected on the dome; the audience is now almost 
immersed in an image surround. 

The idea of sitting inside an image has been used in the field of aerospace 
sim-ulation for many decades to train pilots and astronauts to safely control 
complex, expensive vehicles through simulated mission environments. More 
recendy, this technology has been adapted for entertainment and educational 
use. "Tour of the Universe" in Toronto and "Star Tours" at Disneyland are 
among the first entertainment applications of simulation technology and virtual 
display environments; about 40 people sit in a room on top of a motion 
platform that moves in synch with a computer-generated and model-based 
image display of a ride through a simulated universe. 
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A.spen Movie Map, done by the M.l.T. Architecture Machine Group in the late 
1970s [Lippman, 1980]. Imageryof the townof Aspen, Colorado, wasshotwith 
i special camera system mounted on top of a car, filming down every street 
and around every corner in town, combined with shots above town from 
cranes, he-licopters, and airplanes and also with shots inside buildings. The 
Movie Map 2;ave the operators the capability of sitting in front of a touch- 
sensitive display screen and driving through the town of Aspen at their own 
rate, taking any route they chose, by touching the screen, indicating what 
turns they wanted to make and what buildings they wanted to enter. In one 
configuration, this was set up so that the operator was surrounded by front, 
back, and side-looking camera im-agery so that, again, they were completely 
immersed in a virtual environment. 

Another notable virtual environment developed for entertainment applica- 
tions was "Sensorama." This was an elegant prototype of an arcade game 
de-signed by Morion Heilig in the mid-1960s and one of the first examples of 
a multisensory simulation environment that provided more than just visual 
input. When you put your head up to a binocular viewing optics system, you 
saw a first-person-viewpoint, stereo-film-loop of a motorcycle ride through New 
York City and you heard three-dimensional binaural sound that gave you 
sounds of the city of New York and of the motorcycle moving through it. As you 
leaned your arms on the handlebar platform built into the prototype and sat in 
the seat, simulated vibration cues were presented. The prototype also had a 
fan for wind simulation that combined with a chemical smell bank to blow 
simulated smells in your face. 

Conceptual versions of virtual environments have been described by 
science fiction writers for many decades. One concept has been called 
telepresence, a technology that would allow remotely situated operators to 
receive enough sen-sory feedback to feel like they are really at a remote 
location and are able to do dif-ferent kinds of tasks. Arthur Clarke has 



described "personalized television safaris" in which the operator could 
virtually explore remote environments with-out danger or discomfort. Heinlein's 
"waldoes" were similar, but were able to ex-aggerate certain sensory 
capabilities so that the operator could, for example, control a huge robot. 
Since 1950, technology has gradually been developed to make telepresence 
a reality. Research continues at other laboratories, such as the Naval Ocean 
Systems Center in Hawaii and MITI's Tele-existence Project in Tsukuba, 
Japan, to develop improved systems to help humans operate safely and 
effectively in hazardous environments such as undersea or outerspace. 

One of the first attempts at developing a telepresence visual system was 
done by the Philco Corporation in 1958. With this system, an operator could 
see an image from a remote camera on a CRT mounted on his head in front of 
his eyes 
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The technology has been moving gradually toward lower-cost personal 
simulation environments in which viewers are also able to control their own 
view-point or motion through a virtual environment. An early example of this 
is the and could control the camera's viewpoint by moving his head [Comeau, 
1961]. A variation of the head-mounted display concept was done by Ivan 
Sutherland in the late 1960s, first at M.l.T's Draper Lab in Cambridge, 
Massachusetts, and then later at the University of Utah [Sutherland, 1968]. 
This helmet-mounted display had a see-through capability so that computer- 
generated graphics could be viewed superimposed onto the real 
environment. As the viewer moved around, the computer-generated objects 
would appear to be stable within that real environment and could be 
manipulated with various input devices. 

The Virtual Environment Workstation project at NASA Ames (VIEW) is 
based upon yet another kind of helmet-mounted display technology and, in 
ad-dition, provides auditory, speech, and gesture interaction within a virtual 
environment. 

the ames virtual environment workstation 

The first virtual environment display system developed at Ames was literally a 
motorcycle helmet with a visor attachment containing two small liquid crystal 
display screens of 100-by-IOO-pixel resolution. Several iterations of this first 
pro-totype have resultad in a much lighter, less claustrophobic headset, with 
a long-term goal of designing a visor-like display package worn on the head. We 



are currently assembling five versions of this third prototype for use in various 
re-search environments. Included on the headset is a microphone for 
connected speech recognition, earphones for 3-D sound cueing, and a head- 
tracking device, as well as the package to hold the LCD display screens and 
viewing optics for each eye. 

The liquid crystal display screens are inexpensive, fiat panel displays that 
draw very little power and that have been used specifically for safety reasons 
and for their lightness. The current resolution is 640 pixels by 220 pixels. 
These transmissive displays accept a standard NTSC video signal and 
require some form of backlighting. In the first version, an LED backlight array 
was used, and. currently, very bright, miniature fluorescent tubes are being 
used. Another key part of the display is the very wide-angle optics through 
which the liquid crystal displays are viewed. The objective is to present a 
wide-angle field of view that closely matches human binocular visual 
capabilities rather than just a small win-dow into this 3-D world. These optics 
allow the displays to completely fill your 
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visual fiela of view, thus helping to achieve a sense of presence in the 3-D 
virtual environment. 

On the helmet is a tracking device that tells the host computer where 
you're looking within the 3-D virtual environment. We're now using an 
electromag-netic device that measures where the head is within a magnetic 
field emitted by a source. Azimuth elevation and roll information of the head is 
combined with XYZ position at a resolution of .03 inch and .10 degree 
accuracy. When this information is received by the host computer, it's sent to 
the graphics system or remote camera system so that, as you turn your head, 
the new position is recorded and a new image that matches that position of 
regard is displayed. As you turn your head, new images are being drawn so 
quickly that you feel like you're com-pletely immersed in the virtual 
environment. Another sensing technology that we're investigating may 
provide the ability to track eye position in order to dis-play a 3-D cursor within 
this environment. Eye-tracking will tell us where your eyes are converged in 
3-D space so that, for example, you'll be able to trigger menus with your 
eyes; we'll also be able to present different kinds of depth-of-field information, 
which is yet another depth cue that has been lacking in typi-cal simulation 
display systems to date. 

Virtual environments in the Ames system are synthesized with 3-D computer- 
generated imagery, or are remotely sensed by user-controlled, stereoscopic 
video camera configurations. The computer image system enables high- 
performance, real-time 3-D graphics presentation at resolutions of 640 by 
480 and 1,000 by 1,000 pixels. This imagery is generated at rales up to 30 
iranies per second as re-quired to update image viewpoints in coordination 



with head and limb motion. Dual independent, synchronized display channels 
are used to present disparate imagery to each eye of the viewer for true 
stereoscopic depth cues. For real-time video input of remote environments, two 
miniature CCD video cameras are used to provide stereoscopic imagery. 
Development and evaluation of several head-coupled, remote camera 
platform and gimbal prototypes is in progress to determine optimal hardware 
and control configurations for remotely controlled, free-flying, or telerobot- 
mounted camera systems. 

As this display technology evolved to give a feeling of being surrounded by 
virtual objects or being inside a virtual environment, we also began to develop 
means of interacting with that virtual world — of literally reaching in and touch- 
ing the virtual objects, picking them up, interacting with virtual control panels, 
etc. Earlier research in this area had been done for various applications that 
re-quired manipulation of remote objects with reasonable dexterity and 
feedback to the operator. For example, in 1954, Ralph Mosher at General 
Electric devel- 
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oped the "Handyman" system — a complicated exoskeleton around the operator 
arm that was used to control a remote robotic arm device. Although operational! 
useful, this control system was rather large and unportable. 

We were interested in getting away from such an invasive device and in 1 98 
contracted VPL Research, Inc., to develop a "dataglove" — a lightweight, flexibl 
glove instrumentad with flex-sensing devices to measure the amount of bend ( 
eachjoint of the fingers and the amount of abduction, the splay between fingei 
[Fisher, 1986]. 2 

Information from the glove is transmitted to the host computer to represei 
what the hand is doing at any moment. In the current, commercially availabl 
DataGlove, the amount of bend is measured by light attenuation within filx optic 
bundles covering the fingers, and magnetic tracking devices give absolul position 
and orientation of the hand within the 3-D space. Position and orienta tion 
combined with the finger bend is used to control a graphic model of the han in the 
virtual environment or to control a remote robot hand. With this capabi ity the 
user can pick up and manipulate virtual objects that appear in the sui rounding 
virtual environment [Foley, 1987]. We've also programmed th dataglove so that 
gestures can be recognized, similar to American sign languag» The operator can 
make a particular gesture and the host will recognize it as us< input to trigger a 
particular command or subroutine. For example, pointing Wit one finger moves 
your viewpoint through the computer-generated environmei as if you were flying 
through that space, with the distance between your finger an your body determining 
your velocity; making a fist lets you grab different objectj and using a three-finger 
point invokes a menu floating in visual space that you ca then use to choose other 



subroutines or information displays. In actual use. oj erators can design their 
own set of gestures as preferred. 

Additional work in progress for the glove will provide the capability for ta< tile 
feedback. An array of very small solenoid actuators has been assembled ths will 
present a sense of texture as you touch a virtual object and, for example, you're 
touching an edge of a virtual cube, as you intersect the edge of the cubl one Une 
of the array of solenoids is triggered to present some sense of edgenesi In future 
scenarios, detailed virtual environment databases or tactile sensors o remote 
robot end-ehectors will transmit tactile information to arrays such as thi integrated 
in the DataGlove. A further requirement for tactile feedback to the oj erator is force 
reflection, through which some sense of solidity and surfac boundaries of 
virtual objects is communicated back to the operator. Now, if th hand is closed on 
a virtual object, the fingers will pass right through the objea 



"Virtual InterFace Environments" 265 

terim solutions include the use of auditory feedback to the operator to indicate 
ntact with and forces applied to virtual objects. In the longer term, technology 
required that will constrain the user's fmgers so that they will ciose only to the 
itside boundary of that virtual object; eventually, full arm and body interaction ill 
need to be enabled. 

The auditory channel is another major part of our sensory input and display 
search. The auditory display in the VIEW project is capable of presenting a 
ide variety of binaural sounds to the user via headphones using sound- 
synthesis chnology developed for music synthesizers. The primary function of 
this dis-lay is to provide both discrete and dynamic auditory cues to augment 
or sup-[y information missing from the visual or gestural displays. For example, 
discrete jund cues can signal contact between telerobot end-effectors and 
target ob-cts; alert the operator to attend to information in a data window that is 
currendy ut of the field of view; or indicate successful recognition of a hand 
gesture. Sim-arly, auditory parameters can be dynamically modulated and 
coordinated with ic other subsystem displays to represent, for example, the 
relative positions of ther objects moving within the virtual environment. In the 
real world, we also ave a very good sense of where sounds are coming from 
around us. Additional esearch at Ames has developed an auditory display 
prototype called the Con-olvotron that is capable of synthetically generating 
three-dimensional sound ues in real-time. These cues are presented via 
headphones and are perceived utside of the user's head at a discrete 
distance and direction in the 3-D space urromiding the user. WVien it is 



integrated into tine YIEW system, the postern i the operator can be monitored 
in real-time and the information used to main-lin up to four localized sound 
cues in fixed positions or in motion trajectories elative to the user. This 
capability will further aid the operator's situational wareness by augmenting 
spatial information from the visual display and pro-iding navigational and 
cueing aids from outside the field of view [Wenzel et al., 988]. 

The VIEW system also includes commercially available, speaker-dependent, 
onnected speech-recognition technology that allows the user to give voice 
inputi a natural, conversational formal that cannot be achieved with highly 
con-:rained discrete word recognition systems or through keyboard input. 
TypicalDeech-mediated interactions are requests for display/report of system 
status, istructions for supervisory control tasks, and verbal commands to 
change in-:rface mode or configuration. Taken together, the capabilities of the 
Virtual En-.ronment Display system described enable a wide range of 
applications 
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Telecollaboration 

A major near-term goal for the Virtual Environment Workstation Project is to 
connect at least two of the current prototype interface systems to a common 
virtual environment database. The two users will participate and interact in a 
shared virtual environment, but each will view it from their relative, spatially 
disparate viewpoints. The objective is to provide a collaborative workspace in 
which re-motely located participants can virtually interact with some of the 
nuances of face-to-face meetings while also having access to their personal 
dataspace facility. This could enable valuable interaction between scientists 
collaborating from dif-ferent locations across the country or even between 
astronauts on a space station and research labs on Earth. With full body 
tracking capability, it would also be possible for users to be represented in this 
space by life-size virtual representa-tions of themselves in whatever form they 
choose — a kind of electronic persona. For interactive theater or interactive 
fantasy applications, these sryles might range from fantasy figures to inanimate 
objects, or different figures to different people. Eventually, telecommunication 
networks may develop that will be configured with virtual environment servers 
for users to dial into remotely in order to interact with other virtual proxies. 

Although the current prototype of the Virtual Environment Workstation has 
been developed primarily to be used as a laboratory facility, the componente 
have been designed to be easily replicable for relatively low cost. As the pro- 
cessing power and graphics frame rate on microcomputers quickly 
increases, portable, personal virtual environment systems will also become 



available. The possibilities of virtual realities, it appears, are as limitless as the 
possibilities of re-ality. They can provide a human interface that disappears — 
a doorway to other worlds. 



william bibson 



William Gibsen. Courtesy of Harm Moskowitz. 

"A year here and he si/'// dreamed of cyberspace, hope fading 
nightiy. All the speed he took, all the turns he'd taken and the 
comers he'd cut in Night City, and si/7/ he'd see the matrix \n his 
sleep, bright lattices ofloglc unfolding across that colorless void. " 
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FROM NEUROMANCER, BY WILLIAM GIBSON 
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« In his science fiction noveis, William Gibson's hallucinatory account of cy- 
berspace provided the first social and spatial blueprint for the digital frontier. In 
his 1984 novel Neuromancer-a colorful, disturbing account of our emerging in- 
formation society — he added the word cyberspace to our vocabulary. His writings 
explore the implications of a wired, digital culture, and nave had tremendous in- 
fluence on the scientists, researchers, theorists, and artists working with virtual 
reality. Gibson's notion of an inhabitable, immersive terrain that exists in the con- 
nections between computer networks, a fluid, architectural space that could ex- 
pand endlessly — an invitation to "jack in"to the "digital matrix" — has opened 
the door to a new genre of literary and artistic forms, and has shaped our ex- 
pectations of what is possible jn virtual environments. 

In this essay, Gibson explains that the word cyberspace initially carne to him as 
a term without meaning, that jt "preceded any concept whatever." He tips his hat 
to the "dangerous oid literary gentleman" from East St. Louis, William S. Bur- 
roughs, whose acidic depictions of twentieth-century life inspired Gibson's own 
fictions. He goes on to suggest that in coining the term cyberspace he was apply- 
ing Burroughs's experimental cut-up technique.This entire essay js, jn fact, writ- 



ten in the manner of the Burroughs cut-up — with its sudden leaps from one idea 
to another. It's an appropriate homage to the Beat writer, whose noveis influenced 
not only Gibson but also an entire generation of dystopian writers, filmmakers, 
and artists. 

In Neuromancer, as well as jn his later noveis Count Zero (1987) and Mona Lisa 
Overdrive (1988), Gibson's vision of cyberspace, with its antiheroes who reside in 
the void between the physical world and the network, helped spark an age of the 
posthuman.The cyborgian redefinition of self is staged in such immersive cyber- 
habitats as M U Ds, virtual communities, and on-line chat spaces, where identity 
has become malleable and interchangeable. Gibson's strange, menadng virtual 
world meshed perfectly with the detached, ironic stance of late-twentieth-century 
culture. 

» 

music beams back to base." 

He phases out on a vector of train whistles and the 
one particular steel-engraved slant of winter sun these 
manifestations favor, leaving the faintest tang of Players 
Navy Cut and opening piano bars of East St. Louis, this 
dangerous ofd literary gentleman who sent so many of us 
out, under sealed orders, years ago . . . 
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Inspector Lee taught a new angle — 

Frequencies of silence; blank walls at street level. In the fiat field. We 
became [ield operators. Decoding the lattices. Patrolling the deep faults. Under 
the lights. Machine Dreams. The crowds, swept with con ... Shibuya Times 
Square Pica-dilly. A parked car, an arena of grass, a fountain filled with earth. 
In the hour of the halogen wolves ... The hour remembered. In radio 
silence ... 

[ust a chance operator in the gasoline crack of history, officer ... 

Assembled word cyberspace from small and readily available components 
of language. Neologic spasm: the primal act of pop poetics. Preceded any 
concept whatever. Slick and hollow — awaiting received meaning. 

All I did: folded words as taught. Now other words acerete in the interstices. 

"Gentlemen, that is not now nor will it ever be my concern ..." 

Not what I do. 

I work the angle of transit. Vectors of neon plaza, licensed consumers, acts 
primal and undreamed of... 



The architecture of virtual reality imagined as an accretion of dreams: tattoo 
par-lors, shooting galleries, pinball arcades, dimly lit stalls stacked with damp- 
stained years of men's magazines, chili joints, premises of unlicensed 
denturists, of fire-works and cut bait, betting shops, sushi bars, purveyors of 
sexual appliances, pawnbrokers, wonton counters, love hotels, hotdog stands, 
tortilla faetones, Chinese greengrocers, liquor stores, herbalists, 
chiropractors, barbers, bars. 

These are dreams of commerce. Above them rise intricate barrios, zones 
of more private fantasy... 

Angle of transit sets us down in front of this dusty cardtable in an 
underground mail in the Darwin Free Trade Zone, muzak-buzz of 
seroanalysis averages for California-Oregon, factoids on EBV mutation rates 
and specific translocations at the breakpoint near the c-myc oncogene ... 

Kelsey's second week in Australia and her brother is keeping stubbornly in- 
condo, doing television, looping Gladiator Skull and a new Japanese game 
called Torture Carden. She walks miles of mail that could as easily be Santa 
Barbara again or Singapore, buying British fashion magazines, shoplifting 
Italian eye-shadow; only the stars at night are diherent, Southern Cross, and 
the Chinese boys skim the plazas on carbon-fiber skateboards trimmed with 
neon. 

She pauses in front of the unlicensed vendor, his face notched with pale scars 
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of sun-cancer. He has a dozen cassettes laid out for sale, their plastic casi 
scratched and dusty. "Whole city in there," he says, "Kyoto, yours for a twenty 
She sees the security man, tall and broad, Kevlar-vested, blue-eyed, homing in I 
throw the oid man out, as she tosses the coin on impulse and snatches the 
thiu up, whatever it is, and turns, smiling blankly, to swan past the guard. 
She's a ] censed consumer, untouchable, and looking back she sees the 
vendor squintin) grinning his defiance, no sign of the $20 coin ... 

No sign of her brother when she returns to the condo. She puts on the glass< 
and the gloves and slots virtual Kyoto ... 

Once perfected, communication technologies rarely die out entirely; rather, tht 
shrink to fit particular niches in the global info-structure. Crystal radios hai 
been proposed as a means of conveying optimal seed-planting times to 
isolate agrarian tribes. The mimeograph, one of many recent dinosaurs of 
the urba office-place, still shines with undiminished samisdat potential in the 
century backwaters, the Late Victorian answer to desktop publishing. Banks 
in ui counted Third World villages still crank the day's totals on black 
Burrougb adding machines, spooling out yards of faint indigo figures on long, 



oddly festn curls of paper, while the Soviet Union, not yet sold on throw-away 
new-tech fui has become the last reliable source of vacuum tubes. The eight- 
track tape formj survives in the truckstops of the Deep South, as a medium for 
country music an spoken-word pornography. 

The Street finds its own uses for things — uses the manufacturers never imaj 
ined. The micro-tape recorder, originally intended for on-the-jump executh 
dictation, becomes the revolutionary medium of magnetisdat, allowang the cova 
spread of banned political speeches in Poland and China. The beeper and 
th cellular phone become economic tools in an increasingly competitive 
market i illicit drugs. Other technological artifacts unexpectedly become 
means of con munication ... The aerosol can gives birth to the urban graffitti- 
matrix. Sovu rockers press homemade Nexidisks out of used chest x-rays ... 

Fifteen stones against white sand. 

The sandals of a giant who was defeated by a dwarf. 
A pavilion of gold, another of silver. 
A waterfall where people pray... 

Her mother removes the glasses. Her mother looks at the 
timer. Three houn "But you don't like games, Kelsey..." 
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"It's not a game," tears in her eyes. "It's a city." Her mother puts on the 
jlasses, moves her head from side to side, removes the glasses. 

"I want to go there," Kelsey says. 

"It's different now. Everything changes." 

"I want to go there," Kelsey insists. She puts the glasses back on because 
the 

iook in her mother's eyes frightens her. 

The stones, the white sand: cloud-shrouded peaks, islands in the 
stream ... 

She wants to go there ... 

"The targeted numerais of the ACADEMY LEADER were hypnogogic sigils 
preceding the dreamstate of film." 



Marcos Novak 

"Liquid Architectures in 
Cyberspace" (1991) 



Marcos Novak, Liquid Architectures. © Marcos Novak. Courtesy of 
Marcos Novak. 

"Liquid architecture js an architecture that breathes, pulse leaps as 
one form and lands as another. Liquid architecti is an architecture 
whose form is contingent on the interes of the beholder; jt js an 
architecture that opens to welcon me and closes to defend me; jt is 
an architecture without doors and hallways, where the next room 
js always when need it to be and what I need it to be. " 
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« Marcos Novak describes himself as a "trans-architect," due to his work with 
computer-generated architectural designs, conceived specifically for the virtual 
domain, that do not exist in the physical world. H is immersive, three-dimensional 
creations are responsive to the viewer, transformable though user interaction. Ex- 
ploring the potential of abstract and mathematically conceived forms, Novak has 
invented a set of conceptual tools for thinking about and constructing territories 
in cyberspace. 

In this essay, Novak introduces the concept of "liquid architecture/' a fluid, 
imaginary landscape that exists only jn the digital domain. Novak suggests a 
type of architecture cut loose from the expectations of logic, perspective, and the 
laws of gravity, one that does not conform to the rational constraints of Euclid- 
ean geometries. He views trans-architecture as an expression of the "fourth 
dimension," which incorporates time alongside space among its primary elements. 
Novak's liquid architecture bends, rotates, and mutates jn interaction with the 
person who inhabits it. In liquid architecture, "science and art, the worldly and 
the spiritual, the contingent and the permanent," converge in a poetics of space 
made possible by emerging, virtual reality technologies. 



Novak describes his work as a process of metamorphosis, a "symphony of 
space," in which 3-D constructions have the properties of music, an experience 
ie has since referred to as "navigable music." His is a poetic language that at- 
tempts to describe the indescribable — "cyberspace as habitat of the imagination, 
a habitat for the imagination." » 



[ntroduction 

kVhat is cyberspace? Here is one composite definition: 

Cyberspace is a completely spatialized visualizaron of all information in 
global information processing systems, along pathways provided by 
present and future Communications networks, enabling full copresence 
and interaction of multiple users, allowing input and output from and to the 
full human sensorium, permit-ting simulations of real and virtual realities, 
remote data collection and control through telepresence, and total 
integration and intercommunication with a full range of intelligent producfs 
and environments in real space. 1 

Cyberspace involves a reversal of the current mode of interaction with com- 
uterized information. At present such information is external to us. The idea of 
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cyberspace subverts that relation; we are now within information. In order to 
do so we ourselves must be reduced to bits, represented in the system, and 
in the process become information anew. 

Cyberspace offers the opportunity of maximizing the benefits of separating 
data, information, andform, a separation made possible by digital technology. 
By reducing selves, objects, and processes to the same underlying ground- 
zero representation as binary streams, cyberspace permits us to uncover 
previously invisible relations simply by modifying the normal mapping from 
data to representation. 

To the composite definition above I add the following: Cyberspace is a 
habitat for the imagination. Our interaction with computers so far has primarily 
been one of clear, linear thinking. Poetic thinking is of an entirely different 
order. To locate the difference in terms related to computers: poetic thinking is 
to linear thinking as random access memory is to sequential access memory. 
Everything that can be stored one way can be stored the other; but in the 
case of sequential storage the time required for retrieval makes all but the 
most predictable strate-gies for extracting information prohibitively expensive. 



Cyberspace is a habitat of the imagination, a habitat for the imagination. 
Cyberspace is the place where conscious dreaming meets subconscious 
dreaming, a landscape of rational magic, of mystical reason, the locus and 
triumph of poetr)) over poverty, of "it-can-be-so" over "it-should-be-so." 

The greater task will not be to impose science on poetry, but to restore poetry to 
science. 

This chapter is an investigation of the issues that arise when we consider 
cyberspace as an inevitable development in the interaction of humans with 
computers. To the extent that this development inverts the present 
relationship oi human to information, placing the human within the information 
space, it is an architectural problem; but, beyond this, cyberspace has an 
architecture of iu own and, furthermore, can contain architecture. To repeat: 
cyberspace is architecture; cyberspace has an architecture; and cyberspace 
contains architecture. 

Cyberspace relies on a mix of technologies, some available, some still 
imag-inary. This chapter will not dwell on technology. Still, one brief comment 
is ap propriate here. A great number of devices are being developed and 
tested thai promise to allow us to enter cyberspace with our bodies. As 
intriguing as this maj sound, it flies in the face of the most ancient dream of all: 
magic, or the desire te will the world into action. Cyberspace will no doubt 
have physical aspects; thi visceral has genuine power over us. And though 
one of the major themes of thii 
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essay has to do with the increasing recognition of the physicality of the mind, I 
find it unlikely that once inside we will tolerate such heavy devices for long. 
Gloves and helmets and suits and vehicles are all mechanocybernetic 
inventions that still rely on the major motor systems of the body, and 
therefore on coarse motor coordination, and more importantly, low nerve- 
ending density. The course of invention has been to follow the course of 
desire, with its access to the parts of our bodies that have the most nerve 
endings. When we enter cyberspace we will expect to feel the mass of our 
bodies, the reluctance of our skeleton; but we will choose to control with our 
eyes, fingertips, lips, and tongues, even genitals. 

The trajectory of Western thought has been one moving from the concrete 
to the abstract, from the body to the mind; recent thought, however, has been 
press-ing upon us the frailty of that Cartesian distinction. The mind is a property 
of the body, and lives and dies with it. Everywhere we turn we see signs of this 
recognition, and cyberspace, in its literal placement of the body in spaces 
invented en-tirely by the mind, is located directly upon this blurring boundary, 
this fault. 



At the same time as we are becoming convinced of the embodiment of the 
mind, we are witnessing the acknowledgment of the inseparability of the two in 
another way: the mind affects what we perceive as real. Objective reality itself 
seems to be a construct of our mind, and thus becomes subjective. 

The "reality" that remains seems to be the reality of fiction. This is the 
reality of what can be expressed, of how meaning emerges. The trajectory of 
thought seems to be from concrete to abstract to concrete again, but the new 
concreteness is not that of Truth, but of embodied fiction. 

The difference between embodied fiction and Truth is that we are the 
authors of fiction. Fiction is there to serve our purposes, serious or playful, and 
to the ex-tent that our purposes change as we change, its embodiment also 
changes. Thus, while we reassert the body, we grant it the freedom to change 
at whim, to become iiquid. 

It is in this spirit that the term liquid architecture is offered. Liquid architec- 
ture of cyberspace; liquid architecture in cyberspace. 

part one: cyberspace 
Poetics and Cyberspace 

Well then, before reading poems aloud to so many people, the first thing one 
must do is invoke the duende. This is the only way all of you will succeed at 
the hard 
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task of understanding metaphors as soon as they arise, without depending on 
in-telligence or the critical apparatus, and be able to capture, as fast as it is 
read, the rhythmic design of the poem. For the quality of a poem can never be 
judged on just one reading, especially not poems like these which are fall of 
what I cali "poetic facts" that respond to a purely poetic logic and follow the 
constructs of emotion and of poetic architecture. Poems like these are not 
likely to be under-stood without the cordial help of the duende. 

— Federico Garcia Lorca, Poet in New York 

The duende is a spirit, a demon, invoked to make comprehensible a "poetic 
fact." an "hecho poetico." An "hecho poetico," in turn, is a poetic image that 
is nol based on analogy and bears no direct, logical explanation (Lorca, 
1989). Thu freeing of language from one-to-one correspondence, and the 
parallel invoca-tion of a "demon" that permits access to meanings that are 
beyond ordinary language, permits Lorca to produce some of the most 
powerful and surprisinj poetry ever written. It is this power that we need to 



harness in order to be abh to contend with what William Gibson called the 
"unimaginable complexity" o cyberspace. 
How does this poetry operate? 

Concepts, like subatomic particles, can be thought to have world lines ii 
space-time. We can draw Feynman diagrams for everything that we can 
name tracing the trajectories from our first encounter with an idea to its latest 
incarna tion. In the realm of prose, the world lines of similar concepts are not 
permitte< to overlap, as that would imply that during that time we would be 
unable to dis tinguish one concept from another. In poetry, however, as in the 
realm of quan turn mechanics, world lines may overlap, split, divide, blink out of 
existence. an< spontaneously reemerge. 2 Meanings overlap, but in doing so 
cali forth associa tions inaccessible to prose. Metaphor moves mountains. 
Visualization recon cues contradiction by a surreal and permissive blending 
of the disparate and fa removed. Everything can modify everything: "Green oh 
how I love you green, writes Lorca, "Giantstars offrost/come with the shadow- 
fish/ that leads the wa for dawn. / Thefig tree chafes its wind/with its 
sandpapered branches, /and th mountain, untamed cat, / bristles sour 
maguey spears? 

If cyberspace holds an immense fascination, it is not simply the fascination o 
the new. Cyberspace stands to thought as flight stands to crawling. The root 
o this fascination is the promise of control over the world by the power of the 
wil In other words, it is the ancient dream of magic that finally nears awakening 
inti some kind of reality. But since it is technology that promises to deliver this 
dreara 
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the question of "how" must be confronted. Simply stated, the question is, 
What is the technology of magic? For the answer we must turn not only to 
computer science but to the most ancient of arts, perhaps the only art: poetry. 
It is in poetry that we find a developed understanding of the workings of 
magic, and not only that, but a wise and powerful knowledge of its purposes 
and potentials. Cy-berspace is poetry inhabited, and to navigate through it is to 
become a leaf on the wind of a dream. 

Tools of poets: image and rhythm, meter and accent, alliteration and rhyme, 
tautology, simile, analogy, metaphor, strophe and antistrophe, antithesis, 
balance and caesura, enjambment and closure, assonance and consonance, 
elision and in-flection, hyperbole, lift, onomatopoeia, prosody, trope, tension, 
ellipsis... poetic devices that allow an inflection of language to produce an 
inflection of meaning. By push and pull applied to both syntax and symbol, 
we navigate through a space of meaning that is sensitive to the most minute 
variations in articulation. Poetry is liquid language. 



As difficult as it may sound, it is with operations such as these that we 
need to contend in cyberspace. Nothing less can sufFice. 

I ara in cyberspace. I once again resort to a freer writing, a writing more fluid 
and random. I need to purge a mountain of brown thoughts whose decay 
blocks my way. I seek the color of being in a place where information flies and 
glitters, con-nections hiss and rattle, my thought is my arrow. I combine words 
and occupy places that are the consequence of those words. Every medium 
has its own words, every mixture of words has a potential for meaning. Poets 
have always known this. Now I can mix the words of different media and 
watch the meaning become navigable, enter it, watch magic and music 
merge.... 



Underlying Considerations 

Vi inimal Restriction and Maximal Binding The key metaphor for cyberspace 
is "being there," where both the "being" and the "there" are user-controlled 
variables, and the primary principle is that ofminimal restriction, that is, that it 
is not only desirable but necessary to impose as few restrictions as possible on 
the definition of cyberspace, this in order to allow both case of implementation 
and richness of experience. In addition, maximal bindingimplies that in 
cyberspace anything can be combined with anything and made to "adhere," 
and that it is the responsibility of the user to discern what the implications of the 
combination are for any given circumstance. Of course, defaults are given to get 
things started, but 
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the full wealth of opportunity will only be harvested by those willing and able I 
customize their universe. Cyberspace is thus a user-driven, self-organizing syi 
tern. 

Multiple Representations Cyberspace is an invented world; as a world it n 
quires "physics," "subjects" and "objects," "processes," a full ecology. But sftm it is 
an invented world, an embodied fiction, one built on a fundamental repn 
sentation of our own devising, it permits us to redirect data streams into dine ent 
representations: selves become multiple, physics become variable, cognirio 
becomes extensible. The boundaries between subject and object are conva 
tional and utilitarian; at any given time the data representing a user may be con 
bined with the data representing an object to produce ... what? 

Digital technology has brought a dissociation between data, informatioi form, 
and appearance. Form is now governed by representation, data is a binai stream, 
and information is pattern perceived in the data after the data has be« seen 
through the expectations of a representation scheme or code. A stream i bits, 



initially formless, is given form by a representation scheme, and informado 
emerges through the interaction of the data with the representation; difiera 
representations allow different correlations to become apparent within the san 
body of data. Appearance is a late aftereffect, simply a consequence of mal 
sunken layers of patterns acting upon patterns, some patterns acting as dat 
some as codes. This leads to an interesting question: what is the informaric 
conveyed by the representation that goes beyond that which is in the data itsell If 
a body of data seen one way conveys different information than the same bot of 
data seen another way, what is the additional information provided by one fon that is 
not provided by the other? Clearly, the answer is pattern, that is,perceivi structure. 
And if different representations provide different perceived inform don, how do 
we choose representations? Not only do different representado! provide 
different information, but in the comparison between representatioi new 
information may become apparent. We can thus distinguish two kinds i 
emergent information: intrarepresentational and interrepresentational. 

I substitute the characters on this and the next page of this text with grey scale 
values; two images emerge, pleasingly rhythmical. The gray tone of the letter e i 
stands out, forming snakes along the pages; I apply spline curves to the snakes, 
and, in another space, my text itself is changing — what will it say? Now I 
combine the two pages, and convert the result into a landscape, using the grey 
values to represent height. What snakes were left after the combination of the two 
pages 
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become Serpent Mounds. The Others who were reading my text with me a 
while ago are now flying over this landscape with me, but only I can command 
it to change. Today.... 

'rom Poetics to Architecture We have examined various aspects of 
cyberspace rom a viewpoint that stresses the power of poetic language over 
ordinary, re-luctive language. Poetic language is language in the process of 
making and is best tudied by ciose examination of poetic artifacts, or, better 
yet, by making poetic rtifacts. 

The transition from real space to cyberspace, from prose to poetry, from fact 
) fiction, from static to dynamic, from passive to active, from the fixed in all its 
>rms to the fluid in its everchanging countenance, is best understood by 
exam-ung that human effort that combines science and art, the worldly and 
the spir-ual, the contingent and the permanent: architecture. 



Even as cyberspace represents the acceptance of the body in the realm of 
the iind, it attempts to escape the mortal plane by allowing everything to be 
con-:rted to a common currency of exchange. Architecture, especially 
visionary ar-ihecture, the architecture of the excess of possibility, 
represents the anifestation of the mind in the realm of the body, but it also 
attempts to escape e confines of a limiting reality. The story of both these 
efforts is illuminating, id in both directions. Cyberspace, as a world of our 
creation, makes us con-mplate the possibility that the reality we exist in is 
already a sort of "cyber-»ace," and the difficulties we would have in 
understanding what is real if such jre the case. Architecture, in its strategies 
for dealing with a constraining real-', suggests ways in which the limitations 
of a fictional reality may be sur-aunted. 

Architecture, most fundamentally, is the art of space. There are three 
funda-;ntal requirements for the perception of space: reference, 
delimitation, and )dulation. If any one is absent, space is indistinguishable 
from nonspace, being m nothingness. This, of course, is the fundamental 
observation of categorical ativity. This suggests that cyberspace does not exist 
until a distance can be per-ved between subject and boundary, that is to say, 
until it is delimited and mod-ted. 

A space modulated so as to allow a subject to observe it but not to inhabit it 
isually called sculpture. A space modulated in a way that allows a subject to 
er and inhabit it is called architecture. Clearly, these categories overlap a 
great il: architecture is sculptural, and sculpture can be inhabited. We can 
now draw an association between sculpture and the manner in which 
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we are accustomed to interacting with computers. The interface is a modulate 
information space that remains external to us, though we may create elabora 
spatial visualizations of its inner structure in our minds. Cyberspace, on the otb 
hand, is intrinsically about a space that we enter. To the extent that this space 
wholly artificial, even if it occasionally looks "natural," it is a modulated spac an 
architectural space. But more than asserting that there is architecture withi 
Cyberspace, it is more appropriate to say that cyberspace cannot exist without 
a chitecture, cyberspace is architecture, albeit of a new kind, itself long dreamed 
( 



part Two: liquid architecture in cyberspace 

.,. and we can in our Thought and Imagination contrive perfect Forms of 
Build-ings entirely separate from Matter, by settling and regulating in a certain 
Order, the Disposition and Conjunction of the Lines and Angles. 

— Leon Battista Alberti, The Ten Books of Architecture 



Visionary Architecture: The Excess of Possibility 

Just as poetry differs from prose in its controlled intoxication with meanings 
be found beyond the limits of ordinary language, so visionary architecture < 
ceeds ordinary architecture in its search for the conceivable. Visionary arch in 
ture, like poetry, seeks an extreme, any extreme: beauty, awe, structure, or the b 
of structure, enormous weight, lightness, expense, economy, detail, complex 
universality, uniqueness. In this search for that which is beyond the immedia it 
proposes embodiments of ideas Uiat are both powerful and concise. More of 
than not these proposals are well beyond what can be built. This is not a we 
ness: in this precisely is to be found the poignancy of vision. 

The Space of Art In imagining how information is to be "spatialized" in 
berspace, it is easy to be overwhelmed by the idea of "entering" the computa 
the first place, and to only consider relatively mundane depictions of space: p 
spectival space, graph space, the space of various simple projection systems. I 
manity's library of depictions of space is far richer than that: synchronically i 
diachronically, across the globe and through time, artists have inventad a wa 
of spatial systems. What would it be like to be inside a cubist universe? a hi« 
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glyphic universe? a universe of cave drawings or Magritte paintings? Just as 
al-ternative renditions of the same reality by different artists, each with a 
particular style, can bring to our attention otherwise invisible aspects of that 
reality, so too can different modes of cyberspace provide new ways of 
interrogating the world. 

The development of abstract art by Malevich, Kandinsky, Klee, Mondrian, 
and other early modern artists prefigures cyberspace in that it is an explicit 
turn-ing away from representing known nature. Perhaps artists have always 
inventad worlds — one is reminded how varied the representations used in the 
arts have been — from Chinese watercolors, to Byzantine icons, to the strange, 
conflicting backgrounds of Leonardo — but those worlds usually made 
reference to some familiar reality. Even when that reality was of a cosmic or 



mystical nature, we find an assumption of similarity to the everyday world. 
Modern artists took on the task of inventing entire worlds without explicit 
reference to external reality. Malevich is pertinent: his architectones are 
architectural studies imagined to exist in a world beyond gravity, against 
gravity. Created as an architecture without func-tional program or physical 
constraint, they are also studies for an absolute architecture, in the same 
sense that we speak of absolute music, architecture for the sake of architecture. 
Paul Klee, in his dairies, speaks about being a "god" in a universe of his own 
making. 

The paintings of Max Ernst or, even more so, Hieronymous Bosch come to 
mind, for their ability to create mysterious new worlds. In the works of these 
artists we not only see worlds fashioned out of unlikely combinations of a 
code consisting of familiar elements, but also meaningful crossings of 
expected conceptual and categorical boundaries. 

As with cyberspace, the space of art is architecture, has an architecture 
andcontains architecture. It is architecture in its ability to create a finely 
controlledsense of depth, even within depictions that are inherently two- 
dimensional; it hasan architecture in its compositional structure; and, by 
depiction, it contains architecture. It can serve as a bridge between 
cyberspace and architecture 

Cyberspace Architecture 

Architecture has been earthbound, even though its aspirations have not. 
Buck-minster Fuller remarked that he was surprised that, in spite of all the 
advances made in the technology of building, architecture remained rooted to 
the ground by the most mundane of its functions, plumbing. Rooted by waste 
matter, architecture has nevertheless attempted to fly in dreams and projects, 
Mies and cathe-irals. 
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Architecture has never suffered a lack of fertile dreams. Once, however, in 
times far less advanced technologically, the distance between vision and 
embod-iment was smaller, even though the effort required for that 
embodiment wai often crushing. Most "grand traditions" began with an 
experimental stage oi danger and discovery and did not become fossilized 
until much laten Hard as i) may be for us to fathom, a gothic cathedral was an 
extended experiment often last ing over a century, at the end of which there 
was the literal risk of collapse. Tin dream and the making were one. Curiously, 
the practice of architecture has £>e-come increasingly disengaged from those 
dreams. Cyberspace permits the schisrc that has emerged to be bridged once 
again. 



Cyberspace alters the ways in which architecture is conceived and 
perceived Beyond computer-aided design (CAD), design computing (DC), or 
the devel opment of new formal means of describing, generating, and 
transforming archi tectural form, Cyberspace encodes architectural knowledge 
in a way that indicate; that our conception of architecture is becoming 
increasingly musical, that archi tecture is spatialized music. Computational 
composition, in turn, combines thea new methods with higher-level 
compositional concepts of overall form subject a local and global constraints 
to transform an input pattern into a finished work In principle, and with the 
proper architectural knowledge, any pattern can Ix made into a work of 
architecture, just as any pattern can be made into music. U order for the data 
pattern to qualify as music or architecture it is passed througl compositional 
"filters," processes that select and massage the data according U the 
intentions of the architect and the perceptual capacity of the viewer. Thi 
"adaptive filtering, " to use a neural net term, provides the beginning of the intel 
ligence that constitutes a Cyberspace and not a hypergraph. This, of course 
means that any information, any data, can become architectonic and 
habitable and that Cyberspace and cyberspace architecture are one and the 
same. 

A radical transformation of our conception of architecture and the publi 
domain is implied by cyberspace. The notions of city, square, temple, institution 
home, infrastructure become permanently extended. The city, traditionally thi 
continuous city of physical proximity, becomes the discontinuous city of cul 
tural and intellectual community. Architecture, normally understood in the con 
text of the first, conventional city, shifts to the structure of relationships 
connections and associations that are webbed over and around the simple 
work of appearances and accommodations of commonplace functions. 

I look to my left, and I am in one city; I look to my right, and I am in another. My 
friends in one can wave to my friends in the other, through my having brought 
them together. 
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It is possible to envision architecture nested within architecture. Cyberspace 
itself is architecture, but it also contains architecture, but now without 
constraint as to phenomenal size. Cides can exist within chambers as 
chambers may exist within cides. Since cyberspace signifies the classical 
object yielding to space and relation, all "landscape" is architecture, and the 
objects scattered upon the land-scape are also architecture. Everything that 
was once closed now unfolds into a place, and everything invites one to enter 
the worlds within worlds it contains. 

I am in an empty park. I walk around a tree, and I find myself in a crowded 
chamber. The tree is gone. I cali forth a window, and in the distance see the 
park, leaving. 



Liquid Architecture 

That is why we can equally well reject the dualism of appearance and 
essence. The appearance does not hide the essence, it reveais it; it is the 
essence. The essence of an existent is no longer a property sunk in the 
caviry of this existent; it is the manifest law which presides over the 
succession of its appearances, it is the principle of the series. 

... But essence, as the principle of the series is definitely only the concate- 
nation of appearances; that is, itself an appearance. 

... The reality of a cup is that it is there and that it is not me. We shall inter- 
pret this by saying that the series of its appearances is bound by a principle 
which does not depend on my whim. 

— Jean-Paul Sartre, Being and Nothingness 

The relationship established between architecture and cyberspace so far is 
not A et complete. It is not enough to say that there is architecture in 
cyberspace, nor that that architecture is animistic or animated. Cyberspace 
calis us to consider the difference between animism and animation, and 
animadon and metamorphosis. iVnimism suggests that entities have a "spirit" 
that guides their behavior. Ani-nation adds the capability of change in 
location, through dme. Metamorphosis s change mform, through time or 
space. More broadly, metamorphosis implies :hanges in one aspect of an 
entity as a function of other aspects, continuously or liscontinuously. I use the 
term liquid to mean animistic, animated, metamor-)hic, as well as crossing 
categorical boundaries, applying the cognitively supercharged operations of 
poetic thinking. 

Cyberspace is liquid. Liquid cyberspace, liquid architecture, liquid cities. Liquid 
architecture is more trian kinetic architecture, robotic architecture, anarchi- 
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CAVE System. Illustration by Milana Huang. Courtesy of Daniel Sundm. 
"More than one user will, however, have to interact in the same 
environment if virtual reality is to become an effective 
visualizationtool. " 



« Media artist Daniel Sandin and engineer Thomas DeFanti joined the 
Electronic Visualization Laboratory at the University of Illinois, Chicago, during 
the 1970s, where they combined their talents in computer graphics and video 
imag-ing. Assisted by graduate student Carolina Cruz-Neira, their research jn 
electronic visualization culminated in 1991 with the design and construction of 
the CAVE (Cave Automatic Virtual Environment).This paper describes their 
achievement in advancing the medium of VR by creating a virtual environment 
that joins in-teractive, computer-generated imagery and 3-D audio with physical 
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space — freeing the user from the confinement of the head-mounted display and 
dataglove. 

The suspension of disbelief, so critical to the overall effect of virtual reality, is 
enhanced by the specific qualities of the CAVE's interface, which is, jn fact, a 
small room of about three cubic meters. After entering the room, the user finds 
himself surrounded by projected images that are seamlessly synchronized on 
three walls, as well as on the floor. It js like stepping onto the stage of a virtual 
theater. The participant wears a pair of stereoscopic viewing glasses with built-in 
head-tracking technology, and carries a "magic wand" that provides the means of 
manipulating real-time imagery with the subtle movement of the wrist.The im- 
mersive experience of the CAVE was intended as an allusion to Plato's cave; its 
multiple screens and surround-sound audio evoke the metaphor of a shadowy rep- 
resentation of reality, suggesting how perception is always filtered through .the 
mind's veil of illusion. 

Unlike other systems of virtual reality, the properties of the CAVE are 
enhanced by the interplay between the real and the virtual. Influenced by Myron 
Krueger's installations, the CAVE jmmersant does not experience dislocation 
and disembodiment, but rather is viscerally aware of his or her physical presence 
"on stage" amid the animated imagery and orchestrated sound. Also influenced 
by Krueger, the multiparticipant capability of the CAVE has been extended by 
networking separate CAVE systems together to explore new forms of telematic 
artworks. » 

Xhe media's confidence that virtual reality has already arrived is a little pre- 
mature (they seem to believe that a set of videogame goggles and gloves is all that 
is needed). But as recent experience at the Electronic Visualization Laboratory 
(EVL) of the University of Illinois at Chicago has shown, the technology still 
needs a good deal of work. 



288 Daniel Sandin, Thomas OeFanti, and Carolina Lruz-Neira 

In pursuit of a practical virtual reality (VR) system, researchers at EVL had 
to develop sophisticated software apphcations, and real-time networks to link 
ad-vanced high-speed computers with a variety of high-tech peripherals 
(such as sound synthesizers and location trackers). In so doing, they had to 
solve techni-cal problems that had limited the usefulness of such systems. 

Real Work The benefits make the development worth the effort. When 
perfected, virtual reality systems may enhance how people work and play, 
con-triving comfortable cybernetic environments that enliven and accelerate 



educa-tion and scientific modeling, in addition to devising new forms of 
recreation. However, VR is not child's play, as is shown by the Chicago 
laboratory's strug-gle to get the discrete parts of such systems to function 
flawlessly together in real time. 

The laboratory's virtual reality installation is mostly configured from corar 
mercially available, state-of-the-art equipment. The installation is called the 
Cave, which is a recursive acronym (that is, an acronym that doubles as one of 
its own elements) that stands for Cave Automatic Virtual Environment. 

While the name suggests the system's physical appearance, it is intended 
more strongly as an allusion to the Allegory of the Cave found in Plato's 
Repub-lie. There the Greek philosopher explored the ideas of perception, 
reality, and u-lusion, using the analogy of a person facing the back of a cave 
alive with shadows that are his only basis for his ideas of what real objects are. 

The laboratory's Cave is a new model for the design of virtual reality 
systems, one that offers several advantages over existing models. Unlike users 
of the video-arcade type of virtual reality system, Cave "dwellers" do not need 
to wear hel-mets, which would limit their view of and mobility in the real 
world, nor don bulky gloves and heavy electronics packs, nor be pushed 
about by movement-restricting platforms, to experience virtual reality. Instead, 
they put on a pair of lightweight "glasses" and walk into the Cave, a 27-cubic- 
meter room with an open side and no ceiling. 

The Cave is in fact a partial cube, with the top and one vertical side missing, 
The other three vertical sides are 3-by-3-meter rear-projection screens facing 
the viewer, while the floor is a fourth screen — a front-projection one. The 
screens are wrapped around hard-to-detect cables that form seams at their 
edges. Behind each screen (or, in the case of the floor screen, above it) is a 
video projector thai shines stereo images onto it as the viewer stands inside 
the cube. 

The glasses trick a user's mind into seeing the screen images as three- 
dimensional objects. Each of the glasses' lenses is a shutter made of a 
liquid-crystal material; an electric pulse rapidly turns the material from clear to 
opaque 
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or opaque to clear. In this way, each of the viewer's eyes can be momentarily 
shielded. 

Every sixtieth of a second, a pair of stereoscopically different images is pro- 
jected onto the screen; each of the images that make up the pair (a left-eye 
view and a right-eye view) is displayed for 1/120 second. The crystal shutters 
are syn-chronized so that one clears and closes and the other closes and 
clears 120 times per second. Thus each eye sees a slightly different image, so 
as to create the illu-sion of three dimensions. 



Note that while the glasses alter what the wearer sees on the screens, they 
do not change the way he or she sees real objects (or other people) in the 
environ-ment. Thus, when a computer-generated image of an object is 
projected into the environment, the viewer may be tempted to ask, "Is it live?" 
(Well, one just has to try to grasp the object; for now, at any rate, one's hand 
will pass through a projected image.) 

The stereo glasses also have an electromagnetic transmitter attached to 
them that can be tracked by a location sensor. When the viewer moves, the 
computers that generate the images can determine that a change in viewing 
perspective is needed, and adjust the images appropriately. 

A continuous image across several surfaces can be produced when each 
pro-jector is controlled by a sophisticated computer graphics workstation. 
Each workstation calculates the perspective from each eye position, and sends 
the data to the projectors. 

A hand-held tracking wand with its own location trackers lets the viewer 
manipulate the images projected onto the screen. This the person does by 
locating an imagmary position in virtual reality and interacting with it when 
the wand "touches" an image. For example, a user can move an object from 
one place to an-other, select various options from a menu, or strike images of 
bells and cause them to "ring." 

An importan! ingredient in a virtual environment is sound. To generate the 
right kind of environmental sounds for whatever application is being run, the 
Cave employs a synthesizer with a musical-instrument digital interface (MIDI) 
and up to eight speakers located at the eight corners of the room. 

Masterminds The Cave's virtual world is presided over by four Crimson VGX 
workstations from Silicon Graphics Inc., Mountain View, CA. Each 
workstation, with 256M-byte memory and two 1.6-gigabyte disks, is 
connected to an Electrohome Marquee 8000 projection display. A Silicon 
Graphics Personal Iris serves as a master controller for the system. 

All workstations communicate via a ScramNet optical-fiber network from 
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Systran Corp., Dayton, OH. The network transfers data between these 
system within the 8-ms window needed to synchronize images displayed on 
the Cave 1 four screens. It can accurately and synchronously broadcast data in 
real time i the high speeds necessary. 

The workstations display stereo images by dividing the VGX frame buffi 
into two fields, each of which contains data for the image to be seen by the left 
a right eye; each field equals half the vertical resolution (1280 by 512 pixels). 
Th liquid-crystal-shutter glasses (CrystalEyes from StereoGraphics Corp.. 



Sa Rafael, CA) alternately allow vision in one eye at the field rate of the 
displays-60 Hz for each eye or 120 Hz combined. The glasses are 
synchronized b infrared signais generated by the CrystalEyes controller. 

Without the glasses, the wall screens show double images; with them, th 
viewer gains a striking sense of visual depth and three-dimensional movemenl 
Although the illusion of motion begins, albeit jerkily, in animation at approxi 
mately 10 Hz (or 10 frames per second), highly disparate areas flickered notic* 
ably even at 30 Hz (60 Hz for both eyes). To avoid that effect, the update rate 
wa doubledto60Hz(120Hz). 

Sound is also provided to enhance the illusion of depth. The use of a soun 
board and MIDI synthesizers in conjunction with the display allows the cora 
puter to generate echoes, Doppler effects, and other sounds associated with 
rea three-dimensional environments and objects, and to direct them to the 
appro priate speakers. Thus the system persuades the listener the sound 
"comes from the proper spot on the horizon. 

For the display and sound to create the best illusion of reality, the systei 
should be alert to the viewer's location and direction of gaze. Accordingly, ai 
electromagnetic transponder mounted on the glasses (a Flock of Birds locatioi 
sensor from Ascension Technology Corp., Burlington, VT) monitors where th 
lead viewer's head is inside the Cave and in what direction it is turned. Infol 
mation on viewer location is sent to the controlling workstation, which does th 
computations for image generation and sends the data to the appropriate 
projectors. The hand-held tracking wand also contains a tracking receiver so 
the syitem can determine where the user is holding and pointing it 

Some Advantages While it is much larger than some other approaches, th 
Cave immerses the user in the virtual reality world. Its high-resolution aro 
distortion-free wide-angle views are lacking in other systems. Its horizontal res 
olution OI3840 two-by-six-millimeter pixels across three screens is roughly twic 
that of High-Definition Television (HDTV). The visual acuity is about 20/1 Ifl 
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which is four times better than current head-mounted displays and on a par 
with binocular omni-oriented monitors (Booms). 

The Cave is relatively immune to errors due to head rotation and nodding. 
With head-mounted displays, the projection plane must move as the viewer's 
head moves to look left or right, up or down. Therefore, in head-mounted 
displays, the viewer must wait for the computer to adjust the monitors 
accordingly ifter turning, a lag that may cause a loss of equilibrium and even 
nausea. In the Cave, the projection plane is stable; a glance to the left inside 
the Cave shows the image already on-screen. 



Also, the Cave is a less intrusive interface. A head-mounted display can be 
uncomfortable and disorienting, because the viewer is cut off from the real 
visual jvorld. Someone wearing it is blind to the real-world risks of, say, 
walking into vails or tripping over the cables needed to connect the headset 
back to the com-)uter. A viewer in the Cave can move around at will, because 
the outside world s still visible through the liquid-crystal glasses. Better still, 
because he or she can tul see the outside world, the Cave frees the viewer to 
collaborate more fully. 

More than one user will, however, have to interact in the same environment 
f virtual reality is to become an effective visualizaron tool in fields such as re- 
earch and education. While the head-mounted display and the Boom 
interfaces lo allow multiple users to share an environment, it is only at the high 
cost of du-ilicating the interface hardware. The participants must themselves 
be repre-ented with virtual selves in each other's simulated environments, 
which adds to system's location-tracking and computational workloads. In 
other words, in rder for me to see you in a head-mounted display 
environment, the computer mst calculate in virtual terms physically in relation 
to me and render your image :cordingly on my headset monitors. 

In the Cave, two or more users may look around as they wish. They need 
not e represented virtually, because they represent themselves physically. At 
present, lough, the screen images are shown from the perspective of only one 
person, the >-called lead viewer, who is being tracked. 

ontinuing Challenges Viewer location tracking remains a challenge. A lag 
:tween changes in viewer position and the updating of the screen was 
some-nes noticeable. While less troublesome than the lag experienced with 
head-ounted displays, it still shows up if the viewer moves too quickly. The 
lab is oking into extrapolation techniques to better predict user motion and 
reduce teractive delays to sensor input. More advanced technology may be 
capable of maintaining several perspec- 
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tives separately on-screen, so that each viewer enjoys an individual 
perspective. However, this technological step is not trivial; with each 
additional viewer, the display update rate increases, and more brightness is 
required from the projec-tor. 

Despite these difficulties, the Cave has drawn long unes of eager would-be 
vis-itors at industry shows since it was first demonstrated at the showcase 
event at Siggraph '92, held in Chicago. The applications demonstrated have 
included practical and educational programs, such as molecular biology 
models, superconductor design, fractal mathematics, weather mapping, and 



environmental management. Some programs were simply entertaining. For 
example, some vis-itors had the opportunity to play music with the hand- 
held wand by ringing bells floating around them. 

Other useful applications of the Cave model of virtual reality include scien- 
tific visualization, medicine, architecture, and art. In 1994, the Electronic 
Visualization Laboratory plans to return to Siggraph with an event called 
Vroom (Virtual Reality Room). The intent is to display the latest in head- 
mounted dis-plays, binocular omni-oriented monitor, and Cave technologies. 
The lab is cur-rently asking others for applications to display in the Cave. 

At the same time, EVL is also working on the creation of a "road-show" 
Cave, which would be easier to transport. While this one-of-a-kind working 
model costs about $600,000 in all, it could eventually become the prototype 
for home- or business-based Caves that might cost no more than a "home 
theater" does today. 



Lhar Gavies 

"Changing Space: Virtual 
Reality as an Arena of 
Embodied Being" (1997) 



r eepondfrom Osmose. Courtesy ofChar Davies. 

The medium of A immersive virtual space/or virtual ulity-as it 
js generally known-has intriguing itential as an arena for 
constructing metaphors about ir existential being-in-the- 
world and for exploring wciousness as jt is experienced 
subjectively, as it js '*." 
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« Canadian artist Char Davies established herself as a painter and filmn before 
achieving international recognition for pioneering artworks that en the 
technologies of virtual reality. Her transition to digital media began j mid-1980s, 
when she first explored how three-dimensional computer jmj techniques could 
give depth to the traditional picture plane. In 1987 she ba a founding director of 
Softimage, a 3-D software company that created sp effects for Hollywood films, 
including the industry landmark Jurassic Park.V leading the company's visual 
research team, Davies adopted the Softimage ii jng program for her own artistic 



purposes. These efforts culminated in tN mersive interactive multimedia work 
Osmose, which premiered atthe Museei contemporain de Montreal in 1995 and 
has since been shown in museums an the world. 

In her paintings, Davies had already expressed an interest in deep-sea di and 
for the specific ways that people encounter nature. For Osmose, Davies her team 
of engineers devised an innovative interface that evokes deep-sea ing by 
allowing the user to "float" through a series of virtual worlds.This ir face 
incorporates the head-mounted display and dataglove, as developed by S Fisher at 
NASA.To this ensemble Davies's team added sensors that enable userto 
navigate through breath and balance.This dramatically expanded the in which the 
participant — or an "immersant," to use Davies's term — Inter with and inhabits a 
virtual environment. With Osmose, she introduced the fi tioning of the body as a 
source for gestural commands in human-computer teraction.This enabled the 
jimmersant to experience a computer-generated sp in an immediate, visceral way 
— particularly one that evokes otherworldly sc« of nature or compelling abstract 
settings. As Davies says, her intent is to "n firm the role of the subjectively 
experienced, *felt' body in cyberspace." 

Osmose makes the physical self— rather than the conscious mind — the la for 
user interaction. By doing so, Davies allows us to be affected by a virj space in 
the same subtle way that we are shaped by our unconscious appreh sion of our 
actual, physical environment. As she quotes from Gaston Bachelai The Poetry of 
Space, "By changing space, by leaving the space of one's usual ser bilities ... we 
change our nature." Davies's achievement was to incorporate i intimate, 
emotional territory of the body jnto the encounter with virtual worl Through this 
work she discovered, as Bachelard suggests, that a shift in en ronment can 
trigger powerful emotional and psychological responses. » 
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1 lie medium of "immersive virtual space," or virtual reality — as it is generally 
mown — has intriguing potential as an arena for constructing metaphors 
about aur existential being-in-the-world and for exploring consciousness as it 
is expe-rienced subjectively, as it isfelt. Such environments can provide a new 
kind of "place" through which our minds may float among three-dimensionally 
extended yet virtual forms in a paradoxical combination of the ephemerally 
immaterial with what is perceived and bodily felt to be real. 

My work as an artist explores VR's capacity for refreshing our "ways of see- 
ing" through the design of immersive virtual environments unlike those of our 



usual sensibilities. Osmose (1994-95) is an interactive, "fully immersive," virtual 
environment that uses a stereoscopic head-mounted display, three- 
dimensionally localized interactive sound, and an embodying interface 
driven by the user's breath and balance. There are nearly a dozen "realms" in 
Osmose, metaphorical reconstructions of "nature" as well as philosophical 
texts and software code. The visual dements within these realms are 
semitransparent and translucent. 1 

Since mid-1995, more than five thousand people have been immersed in 
Osmose. The overall effect on immersants appears to be quite complex. 
Many of their responses are surprising in terms of emotional intensity, ranging 
from eu-phoria to tears of loss. The experience of seeing and floating 
through things, along with the work's reliance on breath and balance as well 
as on solitary im-mersion, causes many participants to relinquish desire for 
active "doing" in favor of contemplative "being." In comparing their reactions 
with those generated by psychological research into traditionally induced 
altered states of consciousness, I have come to believe that full-body 
immersion in an "unusual" virtual environment can potentially lead to shifts in 
mental awareness. That this may be pos-sible has many implications, some 
promising, some disturbing. 2 



changing space 

Thirty years ago, in The Poetics of Space, the philosopher Gaston Bachelard 
ex-amined the psychologically transformative potential of "real" environments 
like the desert, the plains, and the deep sea, immense open spaces unlike 
the urban environments to which most of us are accustomed: 

By changing space, by leaving the space of one's usual sensibilities, one 
enters into communication with a space that is psychically innovating. For 
we do not change place, we change our nature. 3 
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Bachelard's poetic insight into the psychological effects of "changing space" is 
echoed by psychologists documenting the effects of traditional contemplative 
practices in terms of altering states of consciousness. According to Arthur Deik- 
man's "Deautomatization and the Mystic Experience," the conditions fostered by 
such practices involve a dehabituating or "deautomatizing" of perceptual sensi-b 
jlides. 

Deautomatization is an undoing of psychic structure permitting the experience 
of increased detail and sensation at the price of requiring more attention. With 
such attention, it is possible that deautomatization may permit the awareness of 



new dimensions of the total stimulus array — a process ol A perceptual expansion" 
... Deautomatization is here conceived as permitting the adult to attain a new, 
fresh perception of the world by freeing him from a stereotyped organization 
built up over the years by allowing adult synthetic functions access to fresh ma- 
terials. The general process of deautomatization would seem of great potential 
usefulness whenever it is desired to break free from an oid pattern in order to 
achieve a new experience ofthe same stimulus or to open a perceptual avenue to 
stimuli never experienced befare. 4 (italics added) 

This dehabituating of perception tends to occur as a result of certain 
psychological conditions, such as when the participant's attention is intensified 
and is directed toward sensory pathways; when there is an absence of controlled, 
an-alytic thought; and when the participant's attitude is one ofreceptivity to stimuli 
rather than defensiveness or suspicion. 5 

Most often attained through rigorous training in age-old meditation tech- 
niques (drug-induced experiences are outside die scope of this essay), such 
conditions lead to an undoing of habitual perceptions — in favor of alternative 
sensibilities. While these may be less efficient in terms of biological or 
psychological survival, psychologists believe that they permit experience of 
aspects of reality previously ignored. The experience of these unusual sensibilities 
includes: 

37. an 
intense sense of "realness," as when inner stimuli become more realthan 
objects 

38. transcendence of time and space 

39. unusual modes of perception 

40. feelings of undiherentiated unity or merging (e.g., a breakdown of distinc- 
tions between things and/or the self and the world) 

41. ineffability or verbal indescribability 
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a profound sense ofjoy or euphoria 

a paradoxical sense of being in and out ofthe body 6 

osmose: breathing in and letting Go 

These "unusual" sensations are eerily similar to what many people claim they 
have experienced during immersion in Osmose. Among the responses we 
have gathered through written comments, correspondence, and video 
interviews, a substantial number of participants have reported the following: 



21. a feeling that they had indeed been somewhere else, in another 
"place" 

2 2. losing track of time (a fifteen-minute session was nearly always 
experiencedas five, a thirty-minute session as ten) 

23. heightened awareness of their own sense of being, or as one 
immersantdescribed it: "as consciousness embodied, occupying space" 7 

24. adeep sense ofmind/body relaxation 

2 5. an inability to speak rationally or put logical words together afterward 

2 6. a feeling of freedom from their physical bodies and an acute 
awareness ofthem at the same time 

27. intense emotional feelings, including euphoria and/or an 
overwhelmingsense of loss when the session was ending, causing some 
to cry and othersto exclaim they were no longer afraid of dying 

In addition, we have observed a pattern of behavior among participants 
during immersion. After becoming accustomed to the interface of breath and 
balance, most people intent on "doing," traveling around to see as much as 
possible in what appears to be an extension of everyday goal-oriented, 
action-based behavior. After ten minutes or so, however, most undergo a 
change: their facial ex-pressions and body gestures loosen, and instead of 
rushing, they slow down, mesmerized by their own perceptions within the 
space. In this final phase, at-tention seems to be directed toward the unusual 
sensations of floating and see-ing through things in what becomes a kind of 
slow-motion perceptual "free-fall." 

What is going on here? 
If these responses are anything to go by, then it appears that immersive 
virtual space, as evidenced by Osmose, can indeed be "psychically 
innovating," to use Bachelard's words. Why? The answer lies in the very nature 
of immersive vir- 
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tual space. Here, ephemeral virtuality coexists with an apparent three- 
dimensionality of form, and feelings of disembodiment can coexist with those o! 
embodiment (given the use of an embodying interface as in Osmose). These para- 
doxical aspects, in combination with the ability to kinesthetically interact with tht 
elements within the space, create a veiy unusual experiential context. 

I want to emphasize, however, that the medium's perception-refreshing po- 
tential is possible only to the extent that a virtual environment is designed to be 
unlike those of our usual sensibilities and assumptions. In Osmose, for example, 
the immersant can unexpectedly see through things and float through them as 
well. Thus the "familiar" becomes the unusual. This creales room for other 
modes of perception: instead of the mind being on autopilot it begins to pay at- 
tention, in the present, lo whal is unusual and unknown. 



osmose: unusual sensibilities 

Osmose does not reconstruct ihe world as we habilually perceive il (as empty 
space containing solid, static, hard-edged, and separate objects, with rigid dis- 
tinctions between subject, object, figure, and ground). Instead, Osmose uses trans- 
parency and luminous particles to "desolidify" things and dissolve spatial 
distinctions. When the immersant moves within the space, multiplicities of semi- 
transparent, three-dimensional forms as well as abstract foreground "flecks'" 
combine to create perceptual ambiguity and slippages between figure and 
ground, near and far, inside and out. Compared to the all too familiar literal rep- 
resentational style commonly associated with three-dimensional computer graph- 
ics, this more evocative aesthetic intensifies the perceptual and cognitive process. 
For the user-interface, a method was developed that relies on the partici-panl's 
own breath and balance rather than on conventional, hand-oriented meth-ods 
such as joystick, wand, trackball, or glove — all of which tend to support a 
distanced and disembodied stance toward the world. This approach, based on 
breathing in to rise, out to fall, and leaning to change direction, brings the expe- 
rience inward, "grounding" it within the core of ihe participant's physical body. 
Conceplually informed by ihe scuba diver's practice of buoyancy control, this 
hands-off technique "frees" participante from the urge to handle and control the 
world of "objects." 8 This approach was intended to "reaffirm" the role of the sub- 
jeclively experienced, "felt" body in cyberspace, in direct contrast to its usual ab- 
sence or objectification in virtual worlds. The use of breathing and balance also 
tends to deeply relax people, creating a tranquil state of mind and body 
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The feelings reported by various participants were probably intensified by the 
solitary nature of the experience, as well as by the fact that the work is "fully im- 
mersive" (its space is perceived as totally enveloping, due to our use of a 
wide-field-of-view head-mounted display). These aspects, in combination 
with the three-dimensionality of the work and the fluid, interactive sound, act to 
amplify the embodied yet virtual nature of the experience. 

While the psychological effects of full-body immersion in a computer- 
generated virtual environment like Osmose have yet to be scientifically 
analyzed, the potential of the medium to dehabituate our sensibilities and 
allow for a re-sensitization of the perception of being invites further 
exploration. 



implications 



Not to be forgotten is the possibility of the medium's potency being used to re- 
place bodily experiences of the "real" with phantasms of virtual Utopias. In her 
preface to Rethinking Technologies, Verona Conley writes about our loss 
of hu-manness" in the wake of the world "becoming technological." 9 

In view of the grim prospect of the twenty-first century, we are compelled to ask how 
critics of culture, philosophers, and artists will deal with technologies. How do they 
contend with expansionist ideology, and the accelerated elimination of diversity and 
of singularities? How do they resist or act? . .. Now, in a world where the notion of 
space has been completely changed through electronic si-multaneity, where the 
computer appears to go faster than the human brain, or where "virtual reality" 
replaces "reality," how do philosophy, critical theory, or artistic practices deal with 
those shifts? 10 

This question aptly applies to immersive virtual space, especially when one 
considers that it will one day likely be used for such (questionable) purposes 
as adapting individuals to psychological and biological survival in a less and 
less "user-friendly" living environment. Moreover, unlike Bachelard's desert or 
deep sea, Deikman's meditation cell, or even an isolation tank, VR is a 
communicative medium, which by default carnes conventional cultural values 
of the Western technoscientific worldview from which that technology has 
sprung. 

The beginnings of an answer to Conley's question may have been 
formulated by Martin Heidegger more than fifty years ago in The Question 
Concerning Technology. As an alternative to what he called technology's 
tendency to function as an instrument of domination and control, Heidegger 
pointed to an earlier form 
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of "techne" called "poiesis" by the Greeks, associated not with "challenging" 
but with a "bringing-forth" or "revealing" into presence. The Greeks 
considered this artistic activity to be somewhat equivalen! to what they called 
physis, or na-ture's own bursting forth of being. 11 1 find inspiration here in terms 
of the use of immersive virtual space as a medium for "bringing forth" or 
"manifesting" ab- • stract ideas into the realm of virtual "place" so that they can 
be kinesthetically ex- j plored and bodily lived. 

This may prove to be a promising use of the medium, and, given effective 
subversion of its culturally bound characteristics, may be a step toward 
suggesting alternative ways of seeing and being in the world. However, even so, 
there remains a significantly disheartening aspect. For even as "places" like 
Osmose may one day be accessible on line as virtual sites of contemplation, 
so too such sites may sig-nal the demise of traditional places of self-reflection 
and tranquillity. In particular this includes "nature" as we know it, as 



compromised in body and habitat by human activities, nature's unfathomable 
presences recede further and further from our urban lives. My own practice in 
the field of "virtual reality" thus con-tains a bittersweet aspect, entangled in 
feelings of both longing and loss. 



1. 



partV 

Narrativity 

William Burroughs 

The Future oFthe Novel" (1964) 
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William Burroughs and Brion Gysin. Copyright© 1978 William 
S. Burroughs andBrion Gysin, reprinted with the permission 
ofThe Wyiie Agency, Inc. 

'Certainly if writing js to have a Mure jt must at least catch 
up with the past and learn to use techniques that have been 
used for some time past jn painting, music and film. " 
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« William Burroughs, appropriately the grandson of the inventor of the ad 
machine, was a catalyst in instigating the literary movement surroundingl 
emerging digital culture jn the 1980s through his surreal mythologies of aliel 
fluence and space-age futurisms.Yet it was Burroughs's preoccupation witfl 
deconstruction of words and language, most notably through the cut-up fold-in 
techniques that he developed with artist Brion Gysin beginning jn H which 
constitutes his most significant contribution to the fragmentary, nonlii approach to 
contemporary narrative — from the cyberpunk novel to the 6 tropically mediated 
hypertext. 

In this essay Burroughs describes the cut-up technique he developed / 
completing his novel Naked Lunch in the late fifties, and later implemented ji 
famous trilogy The Soft Machine, The Ticket That Exploded, and Nova 



Express. His in was to find a way, through writing, to explore nonlinear 
perceptions of space time. Burroughs viewed the cut-up as a way for the writer to 
discover new i nections that extended and ultimately dissolved the rational 
associations of i ventional narrative. Borrowing from the collage technique of 
visual artists, t notably the surrealists, his method links fragments of texts in 
surprising ju positions, offering unexpected leaps into uncharted territories. For 
this rea Burroughs refers to himself as u a map maker, an explorer of psychic 
areas. 

Burroughs's writing experiments of ihe early 1960s express the essei 
narrative strategies of computer-based multimedia storytelling long beforel time. 
He regards the reading experience as one of entering into a multidirecti web of 
different voices, ideas, perceptions, and periods of time. This kind of i igation 
echoes the work of Vannevar Bush, Douglas Engelbart, and Ted Nel who used 
personal association as a guiding principie for the organizaron a formation, and 
proposed hypermedia systems that would make private, ass< tive thinking 
accessible to others. For Burroughs, narrative operates as a j multithreaded 
network that reflects the associative tendencies of the mind. 

In my writing I am acting as a map maker, an explorer of psychic areas, to 
the phrase of Mr Alexander Trocchi, as a cosmonaut of inner space, and I se 
point in exploring areas that have already been thoroughly surveyed — A Rm 
scientist has said: "We will travel not only in space but in time as well — " Tl to 
travel in space is to travel in time — If writers are to travel in space time am 
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plore areas opened by the space age, I think they must develop techniques 
quite as new and definite as the techniques of physical space travel — Certainly 
if writ-ing is to have a future it must at least catch up with the past and learn to 
use techniques that have been used for some time past in painting, music and 
film — Mr Lawrence Durrell has led the way in developing a new form of writing 
with time and space shifts as we see events from different viewpoints and realize 
that so seen they are literally not the same events, and that the oid concepts of 
time and real-ity are no longer valid — Brion Gysin, an American painter living in 
Pans, has used what he calis "the cut-up method" to place at the disposal of 
writers the collage used in painting for fifty years — Pages of text are cut and 
rearranged to form new combinations of word and image — In writing my last 
two noveis, Nova Express and The Ticket That Exploded, I have used an 
extension of the cut-up method I cali "the fold-in method" — A page of text — my 



own or someone else's — is folded down the middle and placed on another 
page — The composite text is then read across half one text and half the other 
— The fold-in method extends to writing the flashback used in films, enabling 
the writer to move backward and forward on his time track — For example I 
take page one and fold it into page one hun-dred — I inseri the resulting 
composite as page ten — When the reader reads page ten he is flashing 
forward in time to page one hundred and back in time to page one — the deja 
vu phenomenon can so be produced to order — This method is of course used 
in music, where we are continually moved backward and forward on the time 
track by repetition and rearrangements of musical themes — 

In using the fold-in method I edit, delete and rearrange as in any other 
method of composition — I have frequently had the experience of writing some 
pages of straight narrative text which were then folded in with other pages 
and found that the fold-ins were clearer and more comprehensible than the 
original texts — Perfectly clear narrative prose can be produced using the fold-in 
method — Best results are usually obtained by placing pages dealing with 
similar subjects njuxtaposition — 

What does any writer do but choose, edit and rearrange material at his 
disposal? — The fold-in method gives the writer literally infinite extension of 
:hoice — Take for example a page of Rimbaud folded into a page of St John 
J erse — (two poets who have much in common) — From two pages an infinite 
lumber of combinations and images are possible — The method could also 
lead o a collaboration berween writers on an unprecedented scale to produce 
works hat were the composite effort of any number of writers living and dead 
— This tappens in fact as soon as any writer starts using the fold-in method 
— I have 
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made and used fold-ins from Shakespeare, Rimbaud, from newspapers, maga- 
zines, conversations and letters so that the noveis I have written using this method 
are in fact composites of many writers — 

I would like to emphasize that this is a technique and like any technique will, of 
course, be useful to some writers and not to others — In any case a matter foi 
experimentation not argument — The conferring writers have been accused by th« 
press of not paying sufficient attention to the question of human survival — Ir 
Nova Express (reference is to an exploding planet) and The Ticket That Exploded, 



I am primarily concerned with the question of survival — with nova conspiracies, 
nova criminals, and nova police — A new mythology is possible in the space ag« 
where we will again have heroes and villains with respect to intentions toward thij 
planet — 



Alian Kaprow 

Untitled Guidelines Por Happenings" (1965) 
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Alian Kaprow, Household. Photo by Sol Goldberg. © Sol 
Goldbarg. 

"The images in each situation can be quite disparate: a kitchen jn 
Hoboken, a pissoir in Paris, a taxi garage in Leopoldville, and a 
bed in some small town in Turkey... None of these planned ties are 
absolutely required, for preknowledge ofthe Happening's cluster 
ofevents by all partidpants will al I ow each one to make his own 
connections. " 
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308 Alian K,aprow 

« Alian Kaprow js best known as a prime mover behind the Happenings. H 
coined the term jn the late 1950s, and led the movement into the bright lights i 
popular culture that characterized the 1960s. Happenings are notoriously difl 
cult to describe, jn part because each was a unique event shaped by the actio» of 
the audience that participated jn any given performance. Simply put, Haj 
penings were held in physical environments — loft spaces, abandoned factorie 
buses, parks, etc. — and brought people, objects, and events jn surprising juxt 
position to one another. With the Happenings, Kaprow rejected the authori and 
primacy of the artwork as an idealized creation and holy act. Rather, he viei art as a 
vehicle for expanding our awareness of life by prompting unexpecte provocative 



interactions. For Kaprow, art is a continual work-in-progress, wi an unfolding 
narrative that is realized through the active participation of the a dience. 

In this essay, Kaprow sets down guidelines for creating Happenings. He t gins 
by dissolving the boundaries between art and everyday life, distributing i 
sponsibility for the execution of the performance to all of jts participants. Jo 
Cage's influence was important to Kaprow's work, particularly the compose 
interest jn indeterminacy, chance operations, and audience involvement. Cacj 
experimental performances at Black Mountain College jn the late 1940s are i 
garded as proto-Happening events. Kaprow's approach actually attempted 
eliminate the distinction between audience and performer all together. 

Kaprow developed techniques to prompt a creative response from the au ence, 
encouraging audience members to make their own connections betwe jdeas and 
events.These narrative strategies relied on a nonlinear sequencing events, and 
the use of indeterminacy to shape the course of the Happenii Kaprow describes 
how a Happening might take place over an extended peri across vast distances, 
or jn many places at once. His techniques reordered pz present, and future within 
a single work jn a remarkable variety of arran ments.The decentralization of 
authorship, location, and narrative — here uni by the intent of the artist and the 
imagination of the participating audiei members — foreshadows nonlinear forms 
in digital media that make use of j nj active and networked technology to expand 
the boundaries of space and time. 



(A) The Une between art and life should be kept as/luid, andperhaps indisti 
as possible. The reciprocity between the man-made and the ready-made wil 
at its maximum potential this way. Something will always happen at this junct 
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which, if it is not revelatory, will not be merely bad art — for no one can easily 
compare it with this or that accepted masterpiece. I would judge this a 
foundation upon which may be built the specific criteria of the Happenings. 

(B) Therefore, the source ofthemes, materiak, actions, and the relationships be- 
tween them are to be derivedfrom anyplace or period exceptfrom the arts, their 
derivatives, and their milieu. When innovations are taking place it often 
becomes 

necessary for those involved to treat their tasks with considerable severity. 
In 



order to keep their eyes fixed solely upon the essential problem, they will 
decide 

that there are certain "don'ts" which, as self-imposed rules, they will obey 
unswervingly. Arnold Schoenberg felt he had to abolish tonality in music com- 
position and, for him at least, this was made possible by evolving the twelve- 
tone 

series technique. Later on his more academic followers showed that it was 
very 

easy to write traditional harmonies with that technique. But still later,John 
Cage 

could permit a C major triad to exist next to the sound of a buzz saw, because 
by 

then the triad was thought of diherently — not as a musical necessity but as 
a 

sound as interesting as any other sound. This sort of freedom to accept all 
kinds 

of subject matter will probably be possible in the Happenings of the future, 
but 

I think not for now. Artistic attachments are still so many window dressings, un- 
consciously held on to to legitimize an art that otherwise might go 
unrecognized. 

Thus it is not that the known arts are "bad" that causes me to say "Don't get 
near them": it is that they contain highly sophisticated habits. By avoiding the 
irtistic modes there is the good chance that a new language will develop that 
has 

ts own standards. The Happening is conceived as an art, certainly, but this is 
for 

ack of a better word, or one that would not cause endless discussion. I, 
person- 

illy, would not care if it were called a sport. But if it is going to be thought of in 
he context of art and artists, then let it be a distinct art which finds its way into 
he art category by realizing its species outside of "culture." A United States Ma- 
ine Corps manual onjungle-fighting tactics, a tour of a laboratory where poly- 
thylene kidneys are made, the daily trafficjams on the Long Island 
Expressway, 

re more useful than Beethoven, Racine, or Michelangelo. 

C) The performance ofa Happening should takeplace overseveral widely 
spaced, nnetimes moving and changing locales. A single performance space 
tends toward ie static and, more significantly, resembles conventional theater 
practice. It is so like painting, for safety's sake, only in the center ofa canvas. 
Later on, when e are used to a fluid space as painting has been for almost a 
century, we can re-irn to concentrated areas, because then they will not be 
considered exclusive, is presently advantageous to experiment by gradually 
widening the distances 
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iW Alan Kaprow 

between the events within a Happening. First along several points on a heai 
trafficked avenue; then in several rooms and floors of an apartment house wfo 
some of the activities are out of touch with each other; then on more than o 
street; then in difFerent but proximate cities; finally all around the globe. On I 
one hand, this will increase the tension between the parts, as a poet might 
stretching the rhyme from two lines to ten. On the other, it permits the parta 
exist more on their own, without the necessity of intensive coordination. Re 



tionships cannot help being made and perceived in any human action, and h< 
they may be of a new kind if tried-and-true methods are given up. 

Even greater flexibility can be gotten by moving the locale itself. A Happx ing 
could be composed for a jetliner going from New York to Luxembourg w 
stopovers at Gander, Newfoundland, and Reykjavik, Iceland. Another Happn 
ing would take place up and down the elevators of five tall buildings in midtoi 
Chicago. 

The images in each situation can be quite disparate: a kitchen in Hobokei 
pissoir in Paris, a taxi garage in Leopoldville, and a bed in some small town 
Turkey. Isolated points of contact may be maintained by telephone and letters, 
a meeting on a highway, or by watching a certain television program at an appoinl 
hour. Other parts of the work need only be related by theme, as when all locales p 
form an identical action which is disjomed in timing and space. But none of thi 
planned ties are absolutely required, for preknowledge of the Happening's clus 
of events by all participants will allow each one to make his own connections. TTi 
however, is more the topic of form, and I shall speak further of this shortly. 

(D) Time, which follows closely on space considerations, should be variable and i 
continuous. It is only natural that if there are multiple spaces in which oca 
rences are scheduled, in sequence or even at random, time or "pacing" w 
acquire an order that is determined more by the character of movements witl 
environments than by a fixed concept of regular development and conclusic 
There need be no rhythmic coordination between the several parts of a Ha 
pening unless it is suggested by the event itself: such as when two persons mi 
meet at a train departing at 5:47 P.M. 

Above all, this is "real" or "experienced" time as distinct from concepti time. If 
it conforms to the clock used in the Happening, as above, that is legi mate, 
but if it does not because a clock is not needed, that is equally legitima All of 
us know how, when we are busy, time accelerates, and how, convers* when 
we are bored it can drag almost to a standstill. Real time is always connect with 
doing something, with an event of some kind, and so is bound up wi things 
and spaces. 
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Imagine some evening when one has sat talking with friends, how as the 
con-versation became reflective the pace slowed, pauses became longer, 
and the speakers "felt" not only heavier but their distances from one another 
increased proportionately, as though each were surrounded by great areas 
commensurate with the voyaging of his mind. Time retarded as space 
extended. Suddenly, from out on the street, through the open window a police 
car, siren whining, was heard speeding by, its space moving as the source of 
sound moved from somewhere to the right of the window to somewhere farther 
to the left. Yet it also carne spilling into the slowly spreading vastness of the 



talkers' space, invading the transformed room, partly shattering it, sliding 
shockingly in and about its envelope, nearly dis-placing it. And as in those 
cases where sirens are only sounded at crowded street corners to warn 
pedestrians, the police car and its noise at once ceased and the capsule of 
time and space it had become vanished as abruptly as it made itself felt. Once 
more the protracted picking of one's way through the extended reaches of 
mind resumed as the group of friends continued speaking. 

Feeling this, why shouldn't an artist program a Happening over the course 
of several days, months, or years, slipping it in and out of the performers' 
daily lives. There is nothing esoteric in such a proposition, and it may have the 
distinct advantage of bringing into focus those things one ordinarily does every 
day with-out paying attention — like brushing one's teeth. 

On the other hand, leaving taste and preference aside and relying solely 
on chance operations, a completely unforeseen schedule of events could 
result, not merely in the preparation but in the actual performance; or a 
simultaneously performed single moment; or none at all. (As for the last, the 
act of finding this out would become, by default, the "Happening.") 

But an endless activity could also be decided upon, which would apparently 
transcend palpable time — such as the slow decomposition of a mountain of 
sand-stone.... In this spirit some artists are earnestly proposing a lifetime 
Happening equivalen! to Clarence Schmidt's lifetime Environment. 

The common function of these alternatives is to release an artist from con- 
ventional notions of a detached, closed arrangement of time-space. A picture, 
a piece of music, a poem, a drama, each confined within its respective frame, 
fixed number of measures, stanzas, and stages, however great they may be in 
their own right, simply will not allow for breaking the barrier between art and 
life. And this is what the objective is. 
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[E) Happenings should be performed once only. At least for the time being, this 
re-striction hardly needs emphasis, since it is in most cases the only course 
possi-ale. Whether due to chance, or to the lifespan of the materials 
(especially the 

perishable ones), or to the changeableness of the events, it is highly unlikely 
thj a Happening of the type I am outlining could ever be repeated. Yet many 
of ti Happenings have, in fact, been given four or five times, ostensibly to 
accomrru date larger attendances, but this, I believe, was only a rationalization 
of the wis to hold on to theatrical customs. In my experience, I found the 
practice inadi quate because I was always forced to do that which could be 



repeated, and had I discard countless situations which I felt were marvelous 
but performable onl once. Aside from the fact that repetition is boring to a 
generation brought up o ideas of spontaneity and originality, to repeat a 
Happening at this time is to ai cede to a far more serious matter: compromise 
of the whole concept of Chang When the practical requirements of a situation 
serve only to kill what an artist hj set out to do, then this is not a practical 
problem at all; one would be very pra tical to leave it for something else more 
liberating. 

Nevertheless, there is a special instance of where more than one 
performaru is entirelyjustified. This is the score or scenario which is designed to 
make evei performance significantly different from the previous one. 
Superficially this hj been true for the Happenings all along. Parts have been 
so roughly scored th there was bound to be some margin of imprecision from 
performance to perfo manee. And, occasionally, sections of a work were left 
open for accidentals or ii provisations. But since people are creatures of habit, 
performers always tendc to fall into set patterns and stick to these no matter 
what leeway was given the in the original plan. 

In the near future, plans may be developed which take their cue from gara 
and athletics, where the regulations provide for a variety of moves that make ti 
outcome always uncertain. A score might be written, so general in its instructioi 
that it could be adapted to basic types of terrain such as oceans, woods, cirie 
farms; and to basic kinds of performers such as teenagers, oid people, 
childre matrons, and so on, including insects, animais, and the weather. This 
could 1 printed and mail-ordered for use by anyone who wanted it. George 
Brecht h been interested in such possibilities for some time now. His sparse 
scores rea like this: 

Direction 

Arrange to observe a sign indicating direction of travel. 

3. travel in the indicated direction 
4. travel in another direction 
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But so far they have been distributed to friends, who perform them at their 
discretion and without ceremony. Certainly they are aware of the philosophic al- 
lusions to Zen Buddhism, of the subtle wit and childlike simplicity of the activ- 
ities indicated. Most of all, they are aware of the responsibility it places on the 
performer to make something of the situation or not. 

This implication is the most radical potential in all of the work discussed 
here. Beyond a small group of initiates, there are few who could appreciate 
the moral dignity of such scores, and fewer still who could derive pleasure 
from going ahead and doing them without self-consciousness. In the case of 



those Happenings with more detailed instructions or more expanded action, 
the artist must be present at every moment, directing and participating, for the 
tradition is too young for the complete stranger to know what to do with such 
plans if he got them. 

(F) Itfollows that audiences should be eliminated entirely. All the elements — 
peo-ple, space, the particular materials and character of the environment, time 
— can in this way be integrated. And the last shred of theatrical convention 
disappears. For anyone once involved in the painter's problem of unifying a field 
of divergent phenomena, a group of inactive people in the space of a 
Happening is just dead space. It is no different from a dead area of red paint on 
a canvas. Movements cali up movements in response, whether on a canvas or 
in a Happening. A Happening with only an empathic response on the part of 
a seated audience is not a Happening but stage theater. 

Then, on a human plane, to assemble people unprepared for an event and 
say that they are "participating" if apples are thrown at them or they are 
herded about is to ask very litde of the whole notion of participation. Most of the 
time the response of such an audience is half-hearted or even reluctant, and 
sometimes the reaction is vicious and therefore destructive to the work 
(though I suspect that in numerous instances of violent reaction to such 
treatment it was caused by the latent sadism in the action, which they quite 
rightly resented). After a few years, in any case, "audience response" proves to 
be so predictably pure cliche that anyone serious about the problem should 
not tolerate it, any more than the painter should continue the use of dripped 
paint as a stamp of modernity when it has been adopted by every 
lampshade and Formica manufacturer in the country. 
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I think that it is a mark of mutual respect that all persons involved in a 
Happening be willing and committed participants who have a clear idea what 
they are to do. This is simply accomplished by writing out the scenario or score 
for all and discussing it thoroughly with them beforehand. In this respect it is 
not different 

from the preparations for a parade, a football match, a wedding, or religious 
ser-vice. It is not even different from a play. The one big diherence is that 
while knowledge of the scheme is necessary, professional talent is not; the 
situations in a Happening are lifelike or, if they are unusual, are so 



rudimentary that profes-sionalism is actually uncalled for. Actors are stage- 
trained and bring over habita from their art that are hard to shake off; the same 
is true of any other kind of show-man or trained athlete. The best participants 
have been persons not normally en-gaged in art or performance, but who are 
moved to take part in an activity that is at once meaningful to them in its ideas 
yet natural in its methods. 

There is an exception, however, to restricting the Happening to participants 
only. When a work is performed on a busy avenue, passers-by will ordinarib 
stop and watch, just as they might watch the demolition of a building. These 
are not theater-goers and their attention is only temporarily caught in the 
course o! their normal affairs. They might stay, perhaps become involved in 
some unex-pected way, or they will more likely move on after a few minutes. 
Such persoru are authentic parts of the environment. 

A variant of this is the person who is engaged unwittingly with a performei 
in some planned action: a butcher will sell certain meats to a customer- 
performei without realizing that he is a part of a piece having to do with 
purchasing, coofc ing, and eating meat. 

Finally, there is this additional exception to the rule. A Happening may b< 
scored iorjust watching. Persons will do nothing else. They will watch things 
each other, possibly actions not performed by themselves, such as a bus 
stoppinjto pick up commuters. This would not take place in a theater or arena, 
but anywhere else. It could be an extremely meditative occupation when done 
devotedryjust "cute" when done indifferently. In a more physical mood, the idea 
of calledfor watching could be contrasted with periods of action. Both normal 
tendencieto observe and act would now be engaged in a responsible way. At 
those momentof relative quiet the observer would hardly be a passive member 
of an audiencehe would be closer to the role of a Greek chorus, without its 
specific mearan necessarily, but with its required place in the overall scheme. 
At other momentthe active and observing roles would be exchanged, so that 
by reciprocation thwhole meaning of watching would be altered, away from 
something like spoonfeeding, toward something purposive, possibly intense 
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But so far they have been distributed to friends, who perform them at their 
discretion and without ceremony. Certainly they are aware of the philosophic al- 
lusions to Zen Buddhism, of the subtle wit and childlike simplicity of the activ- 
ities indicated. Most of all, they are aware of the responsibility it places on the 
performer to make something of the situation or not. 

This implication is the most radical potential in all of the work discussed 
here. Beyond a small group of initiates, there are few who could appreciate 
the moral dignity of such scores, and fewer still who could derive pleasure 
from going ahead and doing them without self-consciousness. In the case of 
those Happenings with more detailed instructions or more expanded action, 



the artist must be present at every moment, directing and participating, for the 
tradition is too young for the complete stranger to know what to do with such 
plans if he got them. 

(F) Itfollows that audiences should be eliminated entirely. All the elements — 
peo-ple, space, the particular materials and character of the environment, time 
— can in this way be integrated. And the last shred of theatrical convention 
disappears. For anyone once involved in the painter's problem of unifying a field 
of divergent phenomena, a group of inactive people in the space of a 
Happening is just dead space. It is no different from a dead area of red paint on 
a canvas. Movements cali up movements in response, whether on a canvas or 
in a Happening. A Happening with only an empathic response on the part of 
a seated audience is not a Happening but stage theater. 

Then, on a human plane, to assemble people unprepared for an event and 
say that they are "participating" if apples are thrown at them or they are 
herded about is to ask very litde of the whole notion of participation. Most of the 
time the response of such an audience is half-hearted or even reluctant, and 
sometimes the reaction is vicious and therefore destructive to the work 
(though I suspect that in numerous instances of violent reaction to such 
treatment it was caused by the latent sadism in the action, which they quite 
rightly resented). After a few years, in any case, "audience response" proves to 
be so predictably pure cliche that anyone serious about the problem should 
not tolerate it, any more than the painter should continue the use of dripped 
paint as a stamp of modernity when it has been adopted by every 
lampshade and Formica manufacturer in the country. 

I think that it is a mark of mutual respect that all persons involved in a 
Happening be willing and committed participants who have a clear idea what 
they are to do. This is simply accomplished by writing out the scenario or score 
for all and discussing it thoroughly with them beforehand. In this respect it is 
not different 

from the preparations for a parade, a football match, a wedding, or religious 
ser- 
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« Since he began producing video art jn the early 1970s, Bill Viola ha; plored 
ways to manipulate and restructure our perception of time and s| through 
electronic media. After several years of creating single-channel work videotape; jn 
which he explored the restructuring of time through techniques extreme slow 
motion, jarring edits, and surreal superimpositions, Viola beg« project video onto 
surfaces, which introduced a spatial element to his inst tions. In 1982, the same 
year that Lynn Hershman released the first videodisc, Loma, Viola wrote this 
article about the narrative potential of ii active media. Here he discusses the 
concept of dataspace, a territory of infoi tion in which all data exists in a continual 
present, outside the traditi definitions of time and space, available for use in 
endless juxtapositions. 



Viola arrives at the notion of dataspace by considering the spaces that I been 
constructed over the ages to record cultural history in architectural fi from Greek 
temples to Gothic cathedrals. He compares these "memory pala to the personal 
computer, with its capacity for storage, instant access, aro formation retrieval. 
Cathedrals, Viola says, communicate through their syml ornamentation, 
illustrative paintings, and stained-glass narratives. Similarlj branching pathways 
and forking paths of computer-controlled video works be explored by the viewer 
searching for story and meaning. As Viola explains, ability to enter a nonlinear 
information matrix at any point, to navigate its tents freely in any direction and at 
any speed, introduces a kind of "jdea spj where the only constraint is the limit of 
the human imagination The computei introduced the "next evolutionary step," 
Viola claims, in which ancient me of memory and artistic expression are reborn 
through the fluid processes a formation technologies. » 



will there be condominiums in data space? 

Possibly the most startling thing about our individual existence is that it is i 
tinuous. It is an unbroken thread — we have been living this same moment 
since we were conceived. It is memory, and to some extent sleep, that gives u 
impression of a life of discrete parts, periods, or sections, of certain time 
"highlights." Hollywood movies and the media, of course, reinforce this 
ception. 

If thmgs are perceived as discrete parts or elements, they can be rearran 
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Gaps become most interesting as places of shadow, open to projection. 
Memory can be regarded as a filter (as are the five senses) — it is a device 
implanted for our survival. The curse of the mnemonist is the flood of images 
that are constantly re-playing in his brain. He may be able to demonstrate 
extraordinary feats of recall, but the rest of the banal and the mundane is 
playing back in there too, endlessly. The result can be lack of sleep, 
psychosis, and even willful death, driving some to seek professional 
psychiatric help (and thus become history on the pages of medicaljournals 
and books). 1 This reincarnates one of the curses of early video art — "record 



everything," the saturation-bombing approach to Ufe which made so many 
early video shows so boring and impossible to sit through. Life without editing, 
it seems, is just not that interesting. 

It is only very recently that the ability to forget has become a prized skill. In 
the age of "information overload," we have reached a critical mass that has 
ac-celerated the perfection of recording technologies, an evolution that leads 
back to ancient times. Artificial memory systems have been around for 
centuries. The early Greeks had their walks through temple, 2 and successive 
cultures have re-fined and developed so-called "mnemo-technics" — Thomas 
Aquinas described an elaborate memory scheme of projecting images and 
ideas on places; in 1482 Jacobus Publicius wrote of using the spheres of the 
universe as a memory system; Giulio Gamillo created a "Memory Theater" in 
Italy in the early 1500s; and Giordano Bruno diagrammed his system of 
artificial memory in his work Shad-ows, published in 1582. Frances Yates 
describes this entire remarkable area in her brilliant book The Art of Memory 
(University of Chicago Press, 1966). 

When I was in Japan in 1981,1 visited a festival of the dead at one of the 
most sacred places in the country, Osoresan Mountain. There I saw blind 
female shamen called itako calling back the spirits of the dead for inquiring 
relatives, a centuries-old practice. Until that time I had felt that the large 
Japanese electron-ics companies were way ahead in the development of 
Communications technol-ogy. After witnessing the itako, however, I realized 
they were way behind. Right in their own backyard were people who, without 
the aid of wires or hardware of any sort, have been for ages regularly 
communicating through time and space with ancestors long gone. An 
interesting place at the temple site (which was perched in the surreal 
landscape of an extinct volcanic crater) was a special walk for the visiting 
pilgrims to take along a prescribed trail. The way led from the temple through a 
volcanic wasteland of rockpiles and smoking fissures to the shores of a crater 
lake. It was called "the walk through Hell." The path through the landscape 
and the points along it all had special significance. The itako, to cali up the 
dead, took this "walk through Hell" in their minds, bringing the spirits in 



vice. It is not even different from a play. The one big diherence is that while 
knowledge of the scheme is necessary, professional talent is not; the situations 
in a Happening are lifelike or, if they are unusual, are so rudimentary that 
profes-sionalism is actually uncalled for. Actors are stage-trained and bring 
over habita from their art that are hard to shake off; the same is true of any other 
kind of show-man or trained athlete. The best participants have been persons 
not normally en-gaged in art or performance, but who are moved to take part in 
an activity that is at once meaningful to them in its ideas yet natural in its 
methods. 

There is an exception, however, to restricting the Happening to participants 
only. When a work is performed on a busy avenue, passers-by will ordinarib 



stop and watch, just as they might watch the demolition of a building. These 
are not theater-goers and their attention is only temporarily caught in the 
course o! their normal affairs. They might stay perhaps become involved in 
some unex-pected way or they will more likely move on after a few minutes. 
Such persoru are authentic parts of the environment. 

A variant of this is the person who is engaged unwittingly with a performei 
in some planned action: a butcher will sell certain meats to a customer- 
performei without realizing that he is a part of a piece having to do with 
purchasing, coofc ing, and eating meat. 

Finally, there is this additional exception to the rule. A Happening may b< 
scored iorjust watching. Persons will do nothing else. They will watch things 
each other, possibly actions not performed by themselves, such as a bus 
stoppinjto pick up commuters. This would not take place in a theater or arena, 
but anywhere else. It could be an extremely meditative occupation when done 
devotedryjust "cute" when done indifferently. In a more physical mood, the idea 
of calledfor watching could be contrasted with periods of action. Both normal 
tendencieto observe and act would now be engaged in a responsible way. At 
those momentof relative quiet the observer would hardly be a passive member 
of an audiencehe would be closer to the role of a Greek chorus, without its 
specific mearan necessarily, but with its required place in the overall scheme. 
At other momentthe active and observing roles would be exchanged, so that 
by reciprocation thwhole meaning of watching would be altered, away from 
something like spoonfeeding, toward something purposive, possibly intense 
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along the familiar path, and when they were through, sent them back the sam< 
way. 

The interesting thing about idea spaces and memory systems is that they 
pre-suppose the existence of some sort of place, either real or graphic, which 
has iti own structure and architecture. There is always a whole space, which 
already ex-ists in its entirety, onto which ideas and images can be mapped, 
using only thai portion of the space needed. 



In addition to the familiar model of pre-recorded time unfolding along a 
linear path (as evidenced by many things, from our writing system to the 
thread of magnetic tape playing in a videotape recorder), there is another 
parallel to be linked with modern technology. "Data space" is a term we hear 
in connection with computers. Information must be entered into a computer's 
memory to create a set of parameters, defining some sort of ground, or field, 
where future cal-culations and binary events will occur. In three-dimensional 
computer graphics, this field exists as an imaginary but real chunk of space, a 
conceptual geometry, theoretically infinite, within which various forms may be 
created, manipulated, extended, and destroyed. The graphics display screen 
becomes that mysterious third point of view looking in on this space (we often 
cali it our "mind's eye"), which can be moved about and relocated from any 
angle at will. The catch is thai the space must exist in the computer first, so 
that there is a reference system within which to locate the various 
coordinates of points and lines called into being by the operator. In our brain, 
constantly flickering pulses of neuron firings create a steady-state field onto 
which disturbances and perturbations are regis-tered as percepts and 
thought forms. This is the notion that something is already "on" before you 
approach it, like the universe, or like a video camera which always needs to be 
"video-ing" even if there is only a blank raster ("nothing") to see. Turn it ofF, 
and it's not video anymore. 

When I had my first experience with computer videotape editing in 1976, one 
demand this new way of working impressed upon me has remained 
significanL It is the idea of holism. I saw then that my piece was actually 
finished and in existence before it was executed on the VTRs. Digital 
computers and software tech-nologies are holistic; they think in terms of 
whole structures. Wordprocessors allow one to write out, correct, and 
rearrange the whole letter before typing it. Data space is fluid and temporal, 
hardcopy is for real — an object is born and becomes fixed in time. Chiseling in 
stone may be the ultimate hard copy. 

When I edited a tape with the computer, for the first time in my Ufe I saw that 
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my video piece had a "score," a structure, a pattern that could be written out 
on paper. We view video and film in the present tense — we "see" one frame at a 
time passing before us in this moment. We don't see what is before it and what 
is after it — we only see the narrow slit of "now." Later, when the lights come 
on, it's gone. The pattern does exist, of course, but only in our memory. 
Notation sys-tems have been around since the beginning of history, since 
what we cali history is notation of events in time, i.e., historical "records." 
With speech we have graphic writing systems; with music we have the score. 
They are both symbolic coded systems for the recording and later playback of 



information events in time. Poetry has always had a level that video or film 
cannot approach (at least not yet): the existence of the words on paper A how 
the poem looks, how the words are placed on the page, the gaps, the 
spacing, etc.). The whole poem is there before us, and, starting at the top of 
the page, we can see the end before we actually get there. 

Our cultural concept of education and knowledge is based upon the idea 
of building something up from a ground, from zero, and starting piece by 
piece to put things together, to construct edifices. It is additive. If we 
approach this process from the other direction, considering it to be 
backwards, or subtractive, all sorts of things start to happen. Scientists 
always marvel at nature, at how it seems to be some grand code, with a built- 
in sense of purpose. Discoveries are made which reveal that more and more 
things are related, connected. Everything appears to be aware of itself and 
everything else, all fitting into an interlocking whole. We quite literally carve out 
our own realities. If you want to make a jigsaw puzzle, you must first start with 
the whole image, and then cut it up. The observer, working backwards into the 
system, has the point of view that he or she is building things up, putting it 
together piece by piece. The prophet Mohamed has said, "All knowledge is 
but a single point — it is the ignorant who have multiplied it." 



the whole Is the sum of its parts 

A friend of mine is an ethnomusicologist who spent several years studying 
the gamelan music of Central Java. He was trained in Western music in the 
States, and spent many years working on his own compositions and 
performing with other musicians. One of the most frustrating things about his 
studies injava, he told me, was trying to work on specific parts of songs with 
the gamelan musicians. Once they were at a rehearsal, and after running 
through a piece, he asked 
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them to play only a section from the middle so that he could make sure he got all 
the notes right. This preved to be an impossible request. After a lot of hemming 
and hawing, excuses, and several false starts, he realized that the group just 
could not do it. They insisted on playing the entire piece over again, from 
beginning to end. In Java, the music was learned by rote, from many years of 
observation and imitation, not from written notation. The idea of taking a small 
part out of con-text, or playing just a few bars, simply did not exist. The music was 
learned and conceived as a whole in the minds of the musicians. 



Giulio Paolini, the contemporary Italian artist, made a little-known but fai 
reaching videotape in the mid-seventies. It was bis first and only tape. Workin at 
an experimental video studio in Florence in the eradle of Western art, he, lil many 
other European artists who visited the art/tapes/22 studio, had bis first ei counter 
with video. Instead of simply re-translating into video what he had a ready been 
doing before, as most other artists had done, Paolini intuitivd recognized the 
great power underlying the recording media. He took the slid< of all bis work, 
most of the pieces he had ever made, and recorded them one at time on each 
frame of video. Playing back this tape, the viewer sees 15 years < Paolini's art, bis 
life's work, go by in less than a minute. Poof! It's gone. 

It is slowly becoming clear that structuralism, currently out of fashion in ti 
fashion-conscious, ever shifting spodight of the art world, must be reconsidere» It 
is vital. However, this new structuralism is not the same as the often ova 
intellectualized, didactic, structuralism-for-structuralism's-sake that took cenu 
stage in the art scene over a decade ago (most visibly through the work of e: 
perimental filmmakers). In retrospect, however, the core ideas being expresse 
dien certainly remain importan!, and perhaps could only have emerged in the wj 
they did given that particular place and moment in cultural time. Furdiermor the 
anti-content messages that have been espoused in various fields of art in ti 
twentieth century also continue to merit attention. We have all been made awai 
diat, since the Renaissance, Western eyes have been drawn to Die visual, to ti 
surface appearance of the world. "Realism" carne to mean how something a] 
peared to the eye alone. Looking at the Gothic art before it, along with Asian an so- 
called Primitive or Tribal Art, it is clear that something fundamental is mis ing. 
However, from our viewpoint today, it is also clear that pure structuralis alone is 
no answer either. 

Decadent art is simply an art which is no longer felt or energized, but merely 
denotes, in which there exists no longer any real correspondence between the 
formal and pictorial elements, its meaning, as it were, negated by the weakness 
or 
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incongruity of the pictorial element; but it is often., .far kss conventional than are 
the primitive or classic stages of the same sequence. True art, pure art, never 
enters into competition with the unattainable perfection of the world. 3 

— A. K. Coomaraswamy 



Structure, or form, has always been the basis of the original pictorial art 
of both Europe and the East, but the Middle Ages were the last time when 
both Eu-rope and Asia met on common artistic ground. 

In Western art, the picture is generally conceived as seen in a frame or through a 
window, and so brought towards the spectator; but the Oriental image really ex- 
ists only in our mind and heart and thence is projected or reflected into space. 

The Indian, or Far Eastern icon, carved or painted, is neither a memory image 
nor an idealization, but a visual symbolism, ideal in the mathematical sense.... 
Where European art naturally depicts a moment of time, an arrested action, or 
an effect of light, Oriental art represents a continuous condition. In traditional 
European terms, we should express this by saying that modern European art en- 
deavors to represent things as they are in themselves, Asiatic and Christian art to 
represent things more nearly as they are in God, or nearer their source. 

— A. K. Coomaraswamy 

The idea of art as a kind of diagram has for the most part not made it down 
from the Middle Ages into modern European consciousness. The 
Renaissance was the turning point, and the subsequent history of Western 
art can be viewed as the progressive distancing of the arts away from the 
sacred and towards the profane. The original structural aspect of art, and 
the idea of a "data space" was pre-served through the Renaissance, 
however, in the continued relation between the image and architecture. 
Painting became an architectural, spatial form, which the viewer 
experienced by physically walking through it. The older concept of an idea 
and an image architecture, a memory "place" like the mnemonic temples of 
the Greeks, is carried through in the great European cathedrals and 
palaces, as is the relation between memory, spatial movement, and the 
storage (recording) ofideas. 

Something extraordinary is occurring today, in the 1980s, which ties together all 
these threads. The computer is merging with video. The potential offspring of 
this marriage is only beginning to be realized. Leaping directly into the farther 
fu-ture for a moment, we can see the seeds of what some have described as 
the ultimate recording technology: total spatial storage, with the viewer 
wandering 
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through some three-dimensional, possibly life-sized field of prerecorded or sil 
ulated scenes and events evolving in time. At present, the interactive video dia 
currently on the market have already begun to address some of these possibi 
ties. Making a program for interactive video disc involves the ordering and stru 



turing (i.e., editing) of much more information than will actually be seen by 
i individual when he or she sits down to play the program. All possible pathwai 
or branches, that a viewer ("participant" is a better word) may take through d 
material must already exist at some place on the disc. Entire prerecorded 
sectia of video may never be encountered by a given observen 

Soon, the way we approach making films and videotapes will drastica! 
change. The notion of a "master" edil and "original" footage will disappe 
Editing will become the writing of a software program that will tell the compul 
how to arrange (i.e., shot order, cuts, dissolves, wipes, etc.) the information on /' 
disc, playing it back in the specified sequence in real time or allowing the vie\i 
to intervene. Nothing needs to be physically "cut" or re-recorded at all. Playba 
speed, the cardinal 30 frames a second, will become intelligently variable and th 
malleable, becoming, as in electronic music practice, merely one fundamental £ 
quency among many which can be modulated, shifted up or down, supera 
posed, or interrupted according to the parameters of electronic wave theo 
Different sections can be assigned to play back at specific speeds or reversed; ai 
individual frames can be held still on the screen for predetermined duratioi 
Other sections can be repeated over and over. Different priorities rule how ai 
in what order one lays material down on the "master" (disc). New talents a 
skills are needed in making programs — this is not editing as we know it. It w 
Nikola Tesla, the original uncredited inventor of the radio, who called it "traj 
mission of intelligence." He saw something there that others didn't. After 
these years, video is finally getting "intelligence," the eye is being reattached 
the brain. As with everything else, however, we will find that the limitatia 
emerging lie more with the abilities and imaginations of the producers and use 
rather than in the tools themselves. 

As in the figure/ground shifts described in Gestalt psychology, we are in I 
process oia shift away from the temporal, piece-by-piece approach ofconstru 
ing a program (symbolized by the camera and its monocular, narrow, tunnel- 
vision, single point of view), and towards a spatial, total-field approach of cartn 
out potentially multiple programs (symbolized by the computer and its holia 
software models, data spaces, and infinite points of view). We are proceedi 
from models of the eye and ear to models of thought processes and concept 
structures in the brain. "Conceptual Art" will take on a new meaning. 
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\s we take the first steps into data space, we discover that there have been many 
3revious occupants. Artists have been there befo re. Giulio Camillo's Memory rheater 
(which he actually constructed in wood, calling it a "constructed body md soul") is 
one example. Dante's Divine Gomedy is another. Fascinating relationships between 
ancient and modern technologies become evident. A simple 2xample can be found in 



the Indian Tantric doctrine of the three traditional ex-pressions of the deity: the 
anthropomorphic, or visual, image; the yantra, or geo-metric "energy" diagram; and 
the mantra, or sonic representation through chanting and music. It is interesting to 
note that these are all considered to be equal — simply outward expressions of the 
same underlying thing. In form, this is not unlike the nature of electronic systems: the 
same electronic signal can be an image if fed into a video monitor, an energy diagram if 
fed into an oscilloscope, and a sequence of sounds if fed into an audio system. 

Today, there are visual diagrams of data structures already being used to describe 
the patterns of information on the computer video disc. The most com-mon one is 
called "branching," a temi borrowed from computer science (figure 1). In this system, 
the viewer proceeds from top to bottom in time, and may ei-ther play the disc 
uninterrupted (arrow), or stop at predetermined branching points along the way and 
go off into related material at other areas on the disc for further study (like a form of 
"visual footnoting"). Examples of this system go something like — in a program on the 
desert, the viewer can stop at a point where plants are mentioned, and branch off to 
more detailed material on the various flora of the valley floor, etc. Although it is clear 
how this can enhance our current educational system, freeing students from boring 
and incompetent teachers so they can proceed at their own pace through information 
which now contains movement, dynamic action, and sound in addition to written 
words, artists know that there must be more out there than this. Even though the 
technology is in-teractive, this is still the same oid linear logic system in a new bottle. 

As a start, we can propose new diagrams, such as the "matrix" structure (Figure 1). 
This would be a non-linear array of information. The viewer could enter at any point, 
move in any direction, at any speed, pop in and out at any place. All directions are 
equal. Viewing becomes exploring a territory, traveling through a data space. Of 
course, it would not be the obviously literal one like the Aspen pro-ject. 4 We are moving 
into idea space here, into the world of thoughts and images as they exist in the brain, 
not on some city planner's drawing board. With the in-tegration of images and video 
into the domain of computer logic, we are begin-ning the task of mapping the 
conceptual structures of our brain onto the technology. After the first TV camera 
with VTR gave us an eye connected to a 
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FIGURE 1. Branching Structure, Matrix Structure, Schizo Structure. 



gross form of non-selective memoiy, we are now at the next evolutionary step 
— the area of intelligent perception and thought structures, albeit artificial. 



Finally, we can envision other diagrams/models emerging as artists go 
deepei into the psychological and neurological depths in search of expressions 
for var-lous thought processes and manifestations of consciousness. 
Eventually certain forms of neurosis, so long the creative fuel of the tormented 
artist in the West, may be mapped into the computer disc. We may end up 
with the "schizo" of "spaghetti" model, in which not only are all directions 
equal, but all are not equal (Figure 1). Everything is irrelevant and significant at 
the same time. Viewers may become lost in this Structure and never fmd their 
way out. 

Worlds are waiting to be explored. It is to be hoped that artists will be given 
their share of access to experiment with this exciting new technology. I 
recently had a glimpse of some of the possibilities for art when I met a 
designer who had first encountered computers while working at a large French 
fashion design firm in New York. There, the graphics artist worked at computer 
terminals. With a light-pen, he could draw various designs, working with 
functions of computei memory and data manipulation. Furthermore, his 
terminal was linked to a large databank of fabric designs and images from 
around the world and throughoul history. After completing a sketch, for 
example, he could cali up a seventeenth-century Japanese kimono design, 
look at it or superimpose it with his own idea. Then he could cali up a turn-of- 
the-century European dress pattern, combine that with his design or integrate 
it with the kimono, all the while storing the various stages in memory. When all 
of this was completed and the final design cho-sen, he could then tie into 
other offices in Europe and the Orient right on the same screen. Designers 
could compare notes, get availability data on his fabric from the milis (i.e., 
where is the best silk, who has stock, what is the order time. etc.). All phases 
of his work could occur on the same screen as digital informa- 
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don. He could travel in space (Europe, the Far East), as well as in time (art his- 
tory), all in an instant and available either as written text or visual images. 

Despite the anti-technology altitudes which still persist (some, it should be 
added, for very good reasons), the present generation of artists, filmmakers, and 
video-makers currently in school, and their instructors, who continue to ignore 



computer and video technology, will in the near future find that they have by- 
passed the primary medium, not only of their own fields, but of the entire culture 
as well. It is imperative that creative artists have a hand in the developments 
currently underway. Computer video discs are being marketed as a great new tool 
in training and education. At this moment, there are creative people 
experimenting with the technology, ensuring that innovative and unique 
applications will emerge; but for now, many of the examples return to the boring 
domain of linear logic in the school classroom. The Aspen city map project is 
perhaps one of the more interesting examples of new program formais. We are at 
the beginning, but even so, for the artisl, standard educational logic slruclures 
are jusl nol ihal interesting. Artists have been to different pans of the brain, and 
know quite well ihal ihings don'l always work like ihey lold you in school. 

II is of paramounl imporlance now, as we watch the same education system 
that brought us through school (and the same Communications syslem ihat gave 
us the wonderful world of commercial TV and AM radio) being mapped onto 
these new technologies, ihal we go back and lake a deeper look at some of the 
older systems described in these pages. Artisls nol shackled to ihe fad and fash- 
ion treadmill of ihe arl world, especially ihe arl world of ihe pasl few years, will 
begin lo see ihe new meaning ihal arl history is laking on. As I have begun to oul- 
line in ihis article, the relation between the image and architecture (as in Re- 
naissance arl), the strucluralism of sacred arl (Orienlal, Early Chrislian, and 
Tribal art, with their mandalas, diagrams, icons, and olher symbolic represenla- 
tions, including song, dance, poetry), and artificial memory systems (the first 
recording technologies from the time of the Greeks through the Middle Ages), 
are all areas thai require furlher invesligalion. 

As we continue lo do our dance with lechnology, some of us more willingly than 
others, the importance of turning back towards ourselves, ihe prime mover of this 
lechnology, grows grealer than the importance of any LSI circuit. The sacred art 
of the pasl has unified form, function, and aesthetics around this single ultimate 
aim. Today, developmenl of self must precede development of the technology or 
we will go nowhere — there will be condominiums in data space (it has already 
begun with cable TV). Applications of tools are only reflections of the users — 
chopsticks may be a simple eating utensil or a weapon, depending on who uses 
them. 
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THE PORCUPINE AND THE CAR 



Late one night while driving down a narrow mountain highway, I carne across 
a large porcupine crossing the road up ahead. Fortunately, I spotted him in 
time to bring the car to a stop a short distance from where he was standing. I 
watched him in the bright headlights, standing motionless, petrified at this 
"ciose en-counter of the third kind." Then, after a few silent moments, he 
started to do a strange thing. Staying in his place, he began to move around 
in a circle, emitting a raspy hissing sound, with the quills rising up offhis body. 
He didn't run away. I realized that this dance was actually a move of self- 
defence. I cut the car headlights to normal beams, but he still continued to 
move around, even more furi-ously, casting weird shadows on the trees 
behind. Finally, to avoid giving him a heart attack, and to get home, I cut the 
lights completely and turned offthe en-gine. I watched him in the dim 
moonlight as he stopped his dance and moved off the road. Later, while driving 
off, I realized that he was probably walking proudly away, gloating over how he 
really gave it to that big blinding noisy thing that rushed toward him out of the 
night. I'm sure he was filled with confidence, so pleased with himself that he 
had won, his porcupine world-view grossly inflated as he headed home in the 
darkness. 



Lynn i-iersnman 
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Lynn Hershman, Deep Contad. © Lynn 
Hershman. Courtesy ofLynn Hershman. 

"Traditional narratives are be'mg restructured. As a resali, people 
feel a greater need to personally participate \n the discovery of 
values that affect and order their Uves, to dissolve the division 
that separates them from control, freedom; replacing longing, 
nostalgia and emptiness with a sense ofidentity purpose and 
hope." 
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« Media artist Lynn Hershman divides her work into two categories: B.C. (Be- 
fore Computers) and A.D. (After Digital). The line of demarcation occurrei 
around 1980 as interactive technologies, including personal computers and 
laserdisc players, became commerdally available. In her early performance work! 



and site-specific installations (B.C.), Hershman had begun exploring themes thai 
focused on issues of identity, alienation, and the blurring between reality and fie- 
tion.The introduction of computers into her work (A.D.) was a natural consfr 
quence of her pursuit of these themes. The confluence of these interests, th« 
theatrical nature of her ideas, and the types of explorations suggested by emerg- 
ing interactive technologies led her into groundbreaking creative terrain. She wa< 
one of the few artists to grasp the expressive possibilities of electronic media anc 
to translate its potential into major interactive artworks for the medium. 

In this essay, Hershman discusses two key works from the early 1980s th 
illustrate how she adopted electronic media to explore new narrative strategie: 
that required and exploited viewer interaction The first of these works was Lorm 
(1982), the seminal art videodisc: a labyrinthine journey through the mental 
landscape of an agoraphobic middle-aged woman Trapped in the seclusion of hei 
tiny apartment, the fictional character Lorna is obsessed with media, television, 
and the bizarre objects of her room.The viewer navigates through the branchin? 
structure of the videodisc to make decisions that ultimately determine the fate of 
Lorna's sordid existence. As Hershman points out in this essay, Lorna's pas-sive 
relation to media and ufe is juxtaposed with the viewer's newfound agencj to 
select and reassemble the narrative's branching themes, stories, interpreta-tions, 
and conclusions. 

Hershman describes her new media artworks as exploring "the interaction 
and intimacy of people despite technological separations." In Deep Contad (1984- 
1989), the other work discussed jn this essay, Hershman uses a touch-screen 
interface to suggest that the viewer can reach through the work's glas A surface, 
the computer's "fourth wall. "The user's interactions are incorporate! into the 
unfolding narrative to confront issues of voyeurism, eroticism, and nar-cissism. In 
Deep Contad, "touching" the protagonist Marion becomes an erotic ges-ture, the 
extension of a phantom limb into the illusionary world of virtual storyspace. As 
Hershman puts it, this type of interactivity constitutes a trans-gression of the 
screen, and transports the viewer into virtual reality. Interactior with virtual 
characters (including the viewer's own jmage, which is projected inte the virtual 
space), encourages the viewer to break free from the traditional au-dience role of 
passive recipient by navigating through the work's many branching narrative 
paths. » 
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A fascinating first. Loma is a stream of consciousness collage that requires 
not only an Interactive left brain, to deduce and make logical cholees, but 
also an in-teracuve right brain, to feel what the character's feeling and to 
understand her life. Whatever Lorna's fate, Lynn Hershman has scored one 
for the history books." 

— Video Magazine 

Despite some theories to the contrary, it is presumed that making art is active 
and /iewing art is passive. Radical developments in communication technology, 
such is the marriage of image, sound, text, computers and interactivity, have 
chal-lenged this assumption. The participants o'\ Loma have reported that 
they had the impression of being empowered because they could manipulate 
Lorna's life. The decision process was placed in their hands, literally. The 
media bath of transmitted, pre-structured and edited information that 
surrounds, and some say alienales, people is washed away, hosed down by 
viewer input. Altering the basis for the information exchange is subversive and 
encourages participation cre-ating a different audience dynamic. 

Interactive systems require viewers to react. Choices must be made. As 
technology expands, there will be more permutations available, not only 
between the viewer and the system, but between elements within the system 
itself. Computer systems will eventually reflect the personality of their users. 
However, there is a space between the system and player in which a link, a 
fusion or transplant occurs. Truth and fiction blur. According to Sigmund 
Freud, reality may be lim-ited to perceptions that can be verified through words 
or visual codes. Therefore, perceptions are the drive to action that influence, if 
not control, real events. Per-ceptions become the key to reality. 

Perhaps it was nostalgia that led me to search for an Interactive video 
fantasy, a craving for control, a longing for real-time activities, a drive toward 
direct action. This chronic condition is reputedly a side effect, or for video 
artists an oc-cupational hazard, that results from watching too much 
television. Television is a medium that is by its nature fragmentary, incomplete, 
distanced and unsatisfy-ing, similar to platonic sex. A precondition of a video 
dialogue is that it does not talk back. Rather, it exists as a moving stasis, a 
one-sided discourse, a trick mir-ror that absorbs rather than reflects. 

Loma was developed as a research and development guide, but it is 
gener-ally inaccessible as it was pressed in a limited edition of 20, of which only 
14 now exist. Loma is only occasionally installed in galleries or museums. 
Creating a truly interactive work demands that it exist on a mass scale, 
available and acces-sible to many people. 
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My path to interactive works began not with video, but with performano 
when, in 1971,1 created an alternative identity named Robena Breitmore. He 
decisions were random, only very remotely controlled. Roberla's manipulaled re 
alily became a model for a privale syslem of inleractive performances. Inslead 
o being kept on a disc or hardware, her records were stored as photographs 
aru texis that could be viewed wilhoul predetermined sequences. This allowed 
view ers to become voyeurs into Roberla's hislory. Their inlerpretations 
shifled, de pending on ihe perspeclive and order of ihe sequences. 

Two years afler Roberta's transformation, Lorna, the first interactive ar 
videodisc, was completed. Unlike Robena, who has many advenlures direclly ii 
the environment, Lorna, a middle-aged, fearful agoraphobic, never leaves ha 
tiny aparlmenl. The premise was ihal ihe more she slays home and walches 
lele vision, ihe more fearful she becomes, primarily because she absorbs the 
fright ening messages of advertising and news broadcasls. Because she 
never leava home, ihe objecls in her room lake on a magnificent proportion. 

Every objecl in Lorna's room is numbered and becomes a chapler in her lifi 
that opens into branching sequences. The viewer, or participanl, accesses 
infor malion aboul her pasl, future and personal conflicts via these objects. 
Many im ages on ihe screen are of ihe remote control device Lorna uses to 
chang i television channels. Because ihe viewer uses a nearly identical unit lo 
direcl th< disc aclion, a melaphoric link or poinl of idenlificalion is eslablished 
belween ih< viewer and Lorna. The viewer activates ihe live action and makes 
surrogate decisions for Lorna. Decisions are designed into a branching path. 
Although there are only 17 minutes of moving image on ihe disc, ihe 36 
chaplers can be se-quenced differently and played over a period of time 
lasling several days. There are ihree separale endings lo ihe disc, ihough ihe 
plol has mulliple varialions ihal include being caughl in repeating dream 
sequences, or using multiple sound-lracks, and can be seen backward, 
forward, at increased or decreased speeds, and from several points of view. 

There is no hierarchy in ihe ordering of decisions. II should be noled ihal thia 
idea is not new. It was explored by such artists as Stephen Mallarme, John 
Cage and Marcel Duchamp, particularly through his music. These artists 
pioneered ideas about random advenlures and chance operalions 50 years 
before ihe in-venlion of ihe lechnology that would have allowed them to exploit 
iheir concepta more fully. 

Lorna's passivity, caused by being controlled by the media, is a 
counterpoinl to the direct aclion of the participante. As the branching palh is 
deconstrucled, players become aware of ihe sublle yel powerful effecls of 
fear caused by ihe 
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nedia, and become more empowered, more active. By acting on Lorna's 
behalf ive travel through their own internal labyrinth to our innermost 
transgressions. 

As Interactive technology is increasingly visible in many areas of society the 
political impact is spectacular. Traditional narratives are being restructured. As 
a result, people feel a greater need to personally participate in the discovery of 
values that affect and order their lives, to dissolve the division that separales 
them from control, freedom; replacing longing, nostalgia and emptiness with a 
sense of identity, purpose and hope. 

The second piece, Deep Contad, refers to the players' ability to travel the 
57 different segments into the deepest part of the disc, determining, through 
their own intuition, the route to the centre, while simultaneously trying to find 
and to feel the deepest, most essential parts of themselves. Viewers 
choreograph their own encounters in the vista of voyeurism that is incorporated 
into Deep Contact. 

This piece developed into a collaboration between many people. John Di 
Ste-fano had the difficult task of composing music that would work in modulated 
segments, as well as backward, forward or in slow-motion. Jiri Vsneska 
assisted immeasurably with the shooting and scanning of photographic 
images, and Marion Grabinsky, the leather-clad protagonist, gave the piece 
the erotic appeal so necessary for sexual transgressions. Toyoji Tomita played 
the Zen Master and Demon with equal charm, while the crew of camera 
operators, editors and pro-duction managers added tremendously to the 
success and joy of making this piece. 

This touch-sensitive interactive videodisc installation compares intimacy 
with reproductive technology, and allows viewers to have adventures that 
change their sex, age and personality. Participants are invited to follow their 
instincts as they are instructed to actually touch their "guide" Marion on any 
part of her body. Adventures develop depending upon which body part is 
touched. 

The sequence begins when Marion knocks on the projected video screen 
ask-ing to be touched. She keeps asking until parts of her body, scanned 
and pro-grammed to roiate onto ihe Microlouch screen, aclually are 
louched. For instance, if you touch her head, you are given a choice of TV 
channels, some giv-ing short, but humorous, analytic accounts of 
"reproductive technologies" and their effect on women's bodies while others 
show how women see themselves. The protagonist also talks about 
"extensions" into the screen that are similar to "phantom limbs," so that the 
screen becomes an extension of the participant's hand. Touching the screen 
encourages the sprouting of phantom limbs, virtual connections between 
viewer and image. 

If Marion's torso is touched, the video image on the disc goes to a bar 
where 
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the viewer can select one of three characters, Marion, a Demon or a Voyeur, to 
follow through interactive fiction that has a video component. If viewers touch 
Mar-ion's legs, they enter a garden sequence in which they can follow Marion, 
a Zen Master, an Unknown Path or a Demon. Selections are made via images 
that have been photographically scanned onto the touchscreen. In the 
garden, for example, the image on the touchscreen is a hand that jumps 
forward depending upon selections, and that allows the viewer to follow the 
lines on the hands to differ-ent routes. The participant usually follows a 
character or a segment to a fork in the road. At this point, the disc 
automatically stops, requiring a selection to go left, to go to the right, to return 
to the first segment of the disc, or to repeat the segment just seen. 

At certain instances viewers can see, ciose up, what they have just passed. 
For example, Marion runs past a bush that, examined closely, reveais a spider 
weav-ing a web. This allows new perceptions of the same scene, depending 
upon the speed at which it is seen. In some instances words are flashed on 
the screen for just three frames, forcing the viewer to go back slowly to see 
what was written. At other points lines are spoken backward, forcing the disc to 
be played in reverse. Whilst the Demon and Zen Master are played by the 
same actor, indicating dif-ferent aspects of our personalities, suggesting that 
the same event can appear frightening or enlightening, depending upon its 
context. 

A surveillance camera was programmed via a Fairlight to be switched "on" 
when a cameraman's shadow is seen. The viewer's image instantaneously 
appears on the screen, displacing and replacing the image. This suggests 
"transgressing the screen" being transported into "virtual reality." 
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(1990) 



Roy Ascott Aspects of Gaia: Digital Pathways Across the 
Whole Earth. Courtesy o/Roy Ascott. 

'Telematics... involves the technology of interactions among 
human beings and between the human m'md and artificial systems 
of intelligence and perception. The individual user of networks js 
always potentially involved \n a global net, and the world js always 
potentially in a state of interaction with the individual. " 
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« In the 1980s Roy Ascott presented a body of art based on telecommunica 
tions technologies, or, as he called jt, "telematic art/' at leading-edge Europeai 
exhibitions of art and technology. These included a pivotal show at the 1981 
Venice Biennale, "Art,Technology and Computer Science," and the 1989 debu' of 
his Aspeas of Gala: Digital Pathways across the Whole Earth ai Ars 
Electronica, ir Austria. For the latter project, Ascott invited artists, musicians, 
and scientist from around the world to contribute digital representations of the 
earth (GaiaJ by way of the global telecommunications network. Ascott's 
exploration of telem-atics prefigured the explosion of Internet art in the mid- 
1990s, shortly after th< emergence of the World Wide Web. 

With the title of this essay, Ascott begs the question of whether artistic worl 
that incorporates telecommunications technologies js truly engaging and of qual- 
ity. He observes how the global network is bringing about a "telematic culture' jn 
which participants are "always potentially in a state of interaction/This dynamic 
Communications environment, he notes, will lead to new opportunities foi artistic 
expression. One consequence will be the emergence of the "integratec data 
work," which he defines as a digital artwork that derives jts expressivenes? from a 
"many-to-many" type of interaction. This collective interaction betweet viewers 
shares much with the Happenings of an earlier era. 

Ascott concludes that telematic art encourages the artist to assume the rol< of 
facilitator, rather than be the sole arbiter of the artwork's creation This em- 
powers the participant and enhances the viewer's capacity for creative thoughl 
and action. In telematic art, he explains, the dynamic action between participant! 
js determined by the work's interface.This approach js quite unlike the traditional 
view of the objet d'art, which concentrates on the static surface of the painting of 
sculpture as executed by the inspired artiste. As Ascott explains, "at the interface 
to telematic systems, content js created rather than received." By determining thi 
interface, the artist shapes the participant's particular experience by using net- 
worked interaction to create a form of participatory narrative — a radical op< 
portunity for personal expression unlike any established artistic genre. » 
1 he past decade has seen the two powerful technologies of computing and 
telecommunications converge into one field of operadons that has drawn into 
its embrace other electronic media, including video, sound synthesis, remote 
sens-ing, and a variety of cybernetic systems. These phenomena are 
exerting enor-mous influence upon society and on individual behavior; they 
seem increasingly 
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to be calling into question ihe very nalure of whal il is to be human, to be 
creative, to think and to perceive, and indeed our relationship to each other 
and lo the planel as a whole. The "telematic culture" thai accompanies ihe 
new develop-menls consisls of a sel of behaviors, ideas, media, values, and 
objeclives ihal are significanlly unlike those thai have shaped society since the 
Enlightenmenl. New cullural and scienlific metaphors and paradigms are being 
generated, new mod-els and representations of reality are being invented, 
new expressive means are being manufactured. 

Telemalics is a lerm used lo designale compuler-mediated Communications 
networking involving telephone, cable, and satellite links belween geographi- 
cally dispersed individuals and inslilulions that are interfaced to data- 
processing systems, remole sensing devices, and capacious data-slorage 
banks. 1 II involves ihe lechnology of interaction among human beings and 
between the human mind and artificial systems of intelligence and perception. 
The individual user of net-works is always polenlially involved in a global nel, 
and ihe world is always po-tentially in a state of interaction with ihe individual. 
Thus, across the vasl spread af telematic networks worldwide, ihe quantity of 
data processed and ihe densily }f informalion exchanged is incalculable. The 
ubiquilous efficacy of ihe telem-ilic medium is nol in doubl, bul the question in 
human lerms, from ihe poinl of aew of culture and creativity, is: Whal is the 
conteni? 

This question, which seems to be at the heart of many critiques of art 
nvolving computers and lelecommunications, suggesls deep-sealed fears 
of he machine coming lo dommale ihe human will and of a technological 
formal-sm erasing human conteni and values. Aparl from all ihe parliculars of 
personal istories, of dreams, desires, and anxieties that inform the conteni of 
art's rich spertoire, the question, in essence, is asking: Is there love in the 
telematic em-race? 
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In ihe attempl to extricale human conteni from technological form, the ques- 
on is made more complicated by our increasing lendency as artists to bring 
to- 

;lher imaging, sound, and text systems into interactive environments ihal exploil 
ate-of-lhe-arl hypermedia and that engage the full sensorium, albeit by digital 
eans. Oul of this technological complexily, we can sense ihe emergence of a 
syn- 

esis of ihe arls. The question of conteni musi iherefore be addressed to what 
ight be called the Gesamtdatenwerk — the integraled dala work — and lo its ca- 
icity to engage the intellecl, emolions, and sensibilily of ihe observen 2 Here, 
iwever, more problems arise, since ihe observer in an interactive telematic 
sys- 

n is by definition a participator. In a telematic art, meaning is not something 
:ated by the artist, dislribuled ihrough the network, and received by ihe ob- 
•ver. Meaning is ihe producl of interaction between the observer and the sys- 
tem, the conteni of which is in a state of flux, of endless change and transion tion. 
In this condition of uncertainty and inslabilily, not simply because of I 
crisscrossing interactions of users of the network but because conteni is e 
bodied in data ihat is itself immaterial, it is pure electronic difference, until it I been 
reconstituted at the interface as image, lexl, or sound. The sensory out\ may be 
differenlialed furlher as exisling on screen, as articulated slruclure malerial, as 
architecture, as environmenl, or in virlual space. 

Such a view is in line wilh a more general approach lo art as residing ii 
cultural Communications system rather than in the arl objecl as a fixed seman 
configuraron — a system in which the viewer aclively negoliates for meaning. 3 this 
sense, lelemalic networking makes explicit in ils lechnology and proloct what is 
implicit in all aesthetic experience where that experience is seen as bei as much 
crealive in ihe acl of the viewer's perception as it is in the act of t arlist's 
production. 4 Classical Communications iheory holds, however, ti 
communication is a one-way dispatch, from sender to receiver, in which of 
contingent "noise" in the channel can modify the message (often fiirther confus as 
the meaning) initiated at the source of transmission. 5 This is the model that h the 
artist as sender and iherefore originalor of meaning, ihe artist as creator ai owner 
of images and ideas, the artist as controller of contexl and conteni. It ii model that 
requires, for its completion, the viewer as, at besl, a skilled decod or interpreter 
of the artist's "meaning" or, at worst, simply a passive receplacle such meaning. It 
gives rise lo ihe industry of crilicism and exegesis in whi< those who 
"understand" ihis or thai work of arl explain il lo ihose who are U stupid or 
uneducated to receive its meaning unaided. In this scenario, the artwa and its 
maker are viewed in the same way as ihe world and its creator. The beau and 
truth of both art and the world are "out there" in the world and in ti work of 
art. They are as fixed and immulable as the material universe appea to be. The 
canon of determinism decrees prefigured harmony and compos tion, regulaled 
form and continuity of expression, with unily and clarily assure by a cultural 
consensus and a linguistic uniformity shared by arlisl and publ alike. 



The problem of conlenl and meaning within a lelematic culture gives adde 
poignancy to ihe rubric "Issues of Conlenl" under which this present wriling o 
compulers and art is developed: "issue" is open to a plurality of meanings, no of of 
which is salisfaclory. The melaphor of a semantic sea endlessly ebbing an 
flowing, of meaning constantly in flux, of all words, utlerances, geslures, and in 
ages in a stale of undecidability, tossed to and fro into new collusions and coi 
junctions wilhin a field of human interaclion and negolialion, is found as muc in 
new science — in quanlum physics, second-order cybernelics, 6 or chaology, 7 fe 
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xample — as in art employing telematic concepts or the new literary criticism that 
las absorbed philosophy and social theory into its practice. This sunrise of un- 
ertainty, of a joyous dance of meaning between layers of genre and 
metaphoric ystems, this unfolding tissue woven of a multiplicity of visual codes 
and cultural maginations, was also the initial promise of the postmodern project 
before it disappeared into the domain of social theory leaving only its frail 
corpus of pes-imism and despair. 

In the case of the physicists, the radical shift in metaphors about the world 
md our participation in its creation and redescription mean that science's 
picure window onto reality has been shattered by the very process of trying to 
measure it. John Wheeler uses this analogy succinctly: 

Nothing is more importan! about the quantum principle than this, that it 
destroys the concept of the world as "sitting out there," with the observer 
safely separated from it by a 20-centimeter slab of piate glass. Even to observe 
so minuscule an ob-ject as an electron, he must shatter the glass. He must 
reach in. He must install his chosen measuring equipment. It is up to him 
whether he shall measure po-sition or momentum ... the measurement 
changes the state of the electron. The universe will never afterwards be the 
same. To describe what has happened one has to cross out that oid word 
"observer" and put in its place "participator." In some strange sense the 
universe is a participatory universe. 8 

In the context of telematic systems and the issue of conteni and meaning, 
the •arallel shift in art of the status of "observer" to that of "participator" is 
demon-trated clearly if in accounts of the quantum principle we substitute 
"data" for quanta." Indeed, fmding such analogies between art and physics is 
more than ist a pleasant game; the web of connections between new models 
of theory and ractice in the arts and the sciences, over a wide domain, is so 
pervasive as to sug-est a paradigm shift in our world view, a redescription of 
reality and a recon-jxtualization of ourselves. We begin to understand that 
chance and change, laos and indeterminacy, transcendence and 
transformation, the immaterial and le numinous, are terms at the center of our 
self-understanding and our new vi-ons of reality. How, then, could there be a 
conteni — sets of meanings — con-ined wilhin telematic art when every aspect 
of nelworking in dalaspace is in a ale of Iransformalion and of becoming? 
The very lechnology of compuler lecommunicalions extends the gaze, 
transcends the body, amplifies the mind to unpredictable configurations of 
thought and creativily.... 

I the recent history of Weslern arl, il was Marcel Duchamp who firsl look ihe 
elaphor of ihe glass, of ihe window onlo ihe world, and turned it back on itself 
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to reveal what is invisible. We see in the work known as The Bride Stripped Barr 
by Her Bachelors, Even, or The Large Glass, a field of vitreous reality in which 
en-ergy and emotion are generated from the tension and interaction of male 
and fe-male, natural and artificial, human and machine. 9 Its subject is 
attraction in Charles Fourier's sense, 10 or, we might even say, love. The 
Large Glass, in its transparent essence, always includes both its environment 
and the reflection of the observen Love is contained in this total embrace; all 
that escapes is reason and certainty. By participating in the embrace, the 
viewer comes to be a progenitor of the semantic issue. The glass as "ground" 
has a function and status anticipating that of the computer monitor as a 
screen of operations — of transformations — and as the site of interaction and 
negotiation for meaning. But it is not only through the Glass that we can see 
Duchamp as prophetic of the telematic mode. The very metaphor of 
networking interaction in a field of uncertainty, in which the observer is creator 
and meaning is unstable, is implicit in all his work. Eqt prophetic in the Glass is 
the horizontal bar that joins the upper and lower ] of the work and serves as a 
metaphor for the all-around viewing, the inclusive.; embracing scope of its 
vision. This stands in opposition to the vertical, head-t toe viewing of 
Renaissance space, embodied in the Western pictorial tradition where the 
metaphor of verticality is employed insistently in its monuments ; architecture 
— emblems often as not of aggression, competition, and dominanc always of 
a tunnel vision. The horizontal, on the other hand, is a metaphor for t bird's-eye 
view, the all-over, all-embracing, holistic systems view of structures, re- 
lationships, and events — viewing that can include the ironic, the fuzzy, and dx 
ambiguous. This is precisely the condition of perception and insight to which 
telematic networking aspires.... 

As Communications networks increase, we will eventually reach a point where 
the billions of information exchanges, shuttling through the networks at any om 
time, can create coherence in the global brain, similar to those found in th< 
human brain. 11 

This suggests equally the need for a redescription of human consciousnes 
as it emerges from the developing symbiosis of the human mind and the artificia 
thought of parallel distributed processing (PDP). 12 

If one of the great rituals of emergence into a new world — that of the Ameri 
can Indian Hopi — is centered around the sacred sipapu that connects to the 
Unn derworld of power and transformation, so our emergence into the new 
world o telematic culture similarly calis for celebration at the interface to those 
PDP sv?-tems that can link us with superconnectivity, mind to mind, into a 
new planet community. And just as the Hopi seek to exploit the full measure 
of their 
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pressive means byjoining image, music, chant, and dance into a holistic unity, 
so we too now seek a synthesis of digital modes — image, sound, text, and 
cybernetic structure — by which to recontextualize our own world, that 
numinous whole of all our separate realities. 

The emerging new order of art is that of interactivity, of "dispersed author- 
ship"; 13 the canon is one of contingency and uncertainty. Telematic art encom- 
passes a wide array of media: hypermedia, videotex, telefacsimile, 
interactive video, computer animation and simulation, teleconferencing, text 
exchange, image transfer, sound synthesis, telemetry and remote sensing, 
virtual space, cybernetic structures, and intelligent architecture. These are 
simply broad categories of technologies and methodologies that are 
constantly evolving — bifurcating, joining, hybridizing — at an accelerated rate. 
At the same time, the status of the art object changes. The culturally domi- 
nant objet d'art as the sole focus (the uncommon carrier of uncommon 
conteni) is replaced by the interface. Instead of the artwork as a window 
onto a corn-posed, resolved, and ordered reality, we have at the interface a 
doorway to un-decidabiliry, a dataspace of semantic and material potentiality. 
The focus of the aesthetic shifts from the observed object to the participating 
subject, from the analysis of observed systems to the (second-order) 
cybernetics of observing sys-tems: the canon of the immaterial and 
participatory. Thus, at the interface to telematic systems, conteni is created 
rather than received. By the same token, conteni is disposed of at the inlerface 
by reinserting it, transformed by the process af inleraclion, back inlo ihe 
network for storage, distribution, and eventual trans-formation at the interface 
of other users, at other access nodes across the planet. A telematic network is 
more than the sum of its parts, more than a computer Communications web. 
The new order of perception it constitutes can be called 'global vision," since 
its distributed sensorium and distributed intelligence — ictworked across the 
whole planet as well as reaching remotely into galactic pace and deep into 
quantum leveis of matter — together provide for a holistic, in-egrative viewing of 
structures, systems, and events that is global in its scope. This rtificial 
extension of human intelligence and perception, which the neural nets f PDP 
and sophisticated remote-sensing systems provide, 14 not only amplifies uman 
perception but is in the process of changing it. The transformation is en- rely 
consistent with the overarching ambition of both art and science through-ut 
this century: to make the invisible visible. Even now, it must be recognized lat 
our human cognitive processes (whether involving linear or associative 
lought, imaging, remembering, computing, or hypothesizing) are rarely carried 
it without the computer being involved. 15 A great proportion of the time that 
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we are involved in communicating, learnmg, or being entertamed entails our i 
teraction with telecommunications systems. 16 Similarly, with feeling and sensii 
artificial, intelligent sensors of considerable subtlety are becoming integral 
human interaction with the environment and to the monitoring of both inten 
and external ecologies. Human perception, understood as the product of act 
negotiation rather than passive reception, thus requires, within this evolvi 
symbiosis of human/machine, telematic links of considerable complexity 1 
tween the very diverse nodes of the worldwide artificial reticular sensorium. 

Telematic culture means, in short, that we do not think, see, or feel in isa 
tion. Creativity is shared, authorship is distributed, but not in a way that den 
the individual her authenticity or power of self-creation, as rather crude mod of 
collectivity might have done in the past. On the contrary, telematic culd 
amplifies the individual's capacity for creative thought and action, for more vr 
and intense experience, for more informed perception, by enabling her to p 
ticipate in the production of global vision through networked interaction w 
other minds, other sensibilities, other sensing and thinking systems across i 
planet — thought circulating in the medium of data through a multiplicity of« 
ferent cultural, geographical, social, and personal layers. Networking suppc 
endless redescription and recontextualization such that no language or vis 
code is final and no reality is ultimate. In the telematic culture, pluralism and i 
ativism shape the configurations of ideas — of image, music, and text — that i 
culate in the system. 

It is the computer that is at the heart of this circulation system, and, like 
heart, it works best when least noticed — that is to say, when it becomes invisil At 
present, the computer as a physical, material presence is too much with uj 
dominates our inventory of tools, instruments, appliances, and apparatus as 
ultimate machine. In our artistic and educational environments it is all too soli 
there, a computational block to poetry and imagination. It is not transpan nor 
is it yet fully understood as pure system, a universal transformative mat The 
computer is not primarily a thing, an object, but a set of behaviors, a systi 
actually a system of systems. Data constitute its lingua franca. It is the agent of 
datafield, the constructor of dataspace. Where it is seen simply as a screen j 
senting the pages of an illuminated book, or as an internally lit painting, it i no 
artistic value. Where its considerable speed of processing is used simpl] 
simulate filmic or photographic representations, it becomes the agent of pas! 
voyeurism. Where access to its transformative power is constrained by a tj 
writer keyboard, the user is forced into the posture of a clerk. The electro 
palette, the light pen, and even the mouse bind us to past practices. The po" 
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of the computer's presence, particularly the power of the interface to shape 
lan- A uage and thought, canriot be overestimated. It may not be an exaggeration 
to say that the "content" of a telematic art will depend in large measure on the 
nature Dl the interface; that is, the kind of configurations and assemblies of 
image, sound, and text, the kind of restructuring and articulation of 
environment that telematic interactivity might yield, will be determined by the 
freedoms and fluidity avail-ible at the interface. 

The essence of the interface is its potential flexibility; it can accept and de- 
iver images both fixed and in movement, sounds constructed, synthesized, 
or sampled, texts written and spoken. It can be heat sensitive, body 
responsive, environmentally aware. It can respond to the tapping of the feet, 
the dancer's irabesque, the direction of a viewer's gaze. It not only articulales 
a physical environment with movement, sound, or light; it is an environment, an 
arena of data-space in which a distributed art of the human/computer 
symbiosis can be acted jut, the issue of its cybernetic content. Each individual 
computer interface is an ispect of a telematic unity such that to be in or at any 
one interface is to be in the virtual presence of all the other interfaces 
throughout the network of which it is / part. This might be defined as the 
"holomatic" principie in networking. It is so Decause all the data flowing 
through any access node of the network are equally md at the same time held 
in the memory of that network: they can be accessed, ;hrough cable or 
satellite links, from any part of the planet at any time of day or light, by users 
of the network (who, in order to communicate with each other, lo not need to 
be in the same place at the same time). 

This holomatic principie was well demonstrated during the Venice Biennale 
>f 1986, when the Planetary Network project had the effect of pulling the exhi- 
>ition from its rather elite, centralized, and exclusive domain in Venice and 
tretching it out over the face of the globe; 1 ' the flow of creative data generated 
iy the interaction of artists networking all over the world was accessible every- 
diere. Set within the interactive environment "Laboratory Ubiqua," a wide range 
f telematic media was involved, including electronic mail, videotex, digital- 
nage exchange, slow-scan TV, and computer conferencing. Interactive 
ideodiscs, remote sensing systems, and cybernetic structures were also in- 
luded.... 

A more elaborate and complex multimedia interface was created for the 
pro-:ct Aspects of Gaia: Digital Pathways across the Whole Earth as part of 
the Ars lectronica Festival of Art and Technology in Linz, Austria, in 1989. 18 The 
trans-lission of digital image and sound by file transfer and the computer 
storage of lefacsimile material via modem, by this time economically feasible, 
invited the 
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creation of a more dramatic and engaging environment for public particip 
Invitations to participate in the project were emailed, faxed, or airmai 
"artists, scientists, poets, shamen [sic], musicians, architects, visionaries 
riginal artists of Australia, native artists of the Americas." The subject of th ject 
was the many aspects of the earth, Gaia, seen from a multiplicity of spi 
scientific, cultural, and mythological perspectives. An energizing stream o 
grated digital images, texts, and sounds (a Gesamtdatenwerk) would thei 
stitute a kind of invisible cloak, a digital noosphere that might contribute 
harmonization of the planet. In accessing the meridians at various noda 
ticipants became involved in a form of global acupuncture, their intera 
endlessly transforming and reconstituting the worldwide flow of creative da 

To the objection that such a global vision of an emerging planetary art 
critically euphoric, or that the prospectus of a telematic culture with its Gt 
datenwerk of hypermediated virtual realities is too grandiose, we should p« 
remind ourselves of the essentially political, economic, and social sensibili 
those who laid the conceptual foundations of the field of interactive sys This 
cultural prospectus implies a telematic politic, embodying the featu feedback, 
self-determination, interaction, and collaborative creativity not i the "science of 
government" for which, over 150 years ago, Andre Marie Ai coined the term 
"cybernetics" — a term reinvigorated and humanized by bert Wiener in this 
century. 19 Contrary to the rather rigid determinism an« itivism that have shaped 
society since the Enlightenment, however, these fe will have to accommodate 
notions of uncertainty, chaos, autopoiesis. c» gency, and the second-order 
cybernetics or fuzzy-systems view of a wa which the observer and observed, 
creator and viewer, are inextricably Uni the process of making reality — all our 
many separate realities interacting. . ing, re-forming, and resonating within the 
telematic-noosphere of the plai 

Within these separate realities, the status of the "real" in the phenomen of 
the artwork also changes. Virtual space, virtual image, virtual reality — thi 
categories of experience that can be shared through telematic networks, alii for 
movement through "cyberspace" and engagement with the virtual pre of 
others who are in their corporeal materiality at a distance, physically in j sible or 
otherwise remote. 20 The adoption of a headset, DataGlove, or othe wear can 
make the personal connection to cyberspace — socialization in h reality — 
wherein interaction with others will undoubtedly be experieiu "real," and the 
feelings and perceptions so generated will also be "reaLI passage from real to 
virtual will probably be seamlessjust as social behavi rived from human- 
computer symbiosis is flowing unnoticed into QUI sciousness. 
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But the very ease of transition from "reality" to "virtuality" will cause 
confusion in culture, in values, and in matters of personal identity. It will be the 
role of the artist, in collaboration with scientists, to establish not only new 
creative praxes but also new value systems, new ordinances of human 
interaction and social communicability. The issue of conteni in the planetary 
art of this emerging telematic culture is therefore the issue of values, 
expressed as transient hypothe-ses rather than finalities, tested within the 
immaterial, virtual, hyperrealities of dataspace. Integrity of the work will not be 
judged by the oid aesthetics; no antecedent criteria can be applied to 
network creativity since there is no previous :anon to accommodate it. The 
telematic process, like the technology that em-Dodies it, is the product of a 
profound human desire for transcendence: to be out )f body, out of mind, 
beyond language. Virtual space and dataspace constitute the lomain, 
previously provided by myth and religion, where imagination, desire, md will 
can reengage the forces of space, time, and matter in ihe battle for a new eality. 
The digital matrix that brings all new electronic and optical media into its 
slematic embrace — being a connectionist model of hypermedia — calis for a "con- 
ective criticism." The personal computer yields to the interpersonal computen 
erial data processing becomes parallel distributed processing. Networks link 
icmory bank to memory bank, intelligence to intelligence. Digital image and 
igital sound find their common groundjust as a synthesis ofmodes — visual, 
lacle, textual, acoustic, environmental — can be expected to "hypermediate" 
the jtworked sensibilities of a constellation of global cultures. The digital 
camera — ithering still and moving images from remote sensors deep in space, 
or directed ' human or artificial intelligence on earth, seeking out what is 
unseen, imaging lat is invisible — meets at a point between our own eyes and 
the reticular retina worldwide networks, stretching perception laterally away 
from the tunnel vi->n, from the Cartesian sight lines of the oid deterministic 
era. Our sensory ex-rience becomes extrasensory, as our vision is enhanced 
by the extrasensory vices of telematic perception. The computer deals 
invisibly with the invisible. Drocesses those connections, collusions, systems, 
forces and fields, transfor-tions and transferences, chaotic assemblies, and 
higher orders of organization t lie outside our vision, outside the gross level of 
material perception afforded our natural senses. Totally invisible to our 
everyday unaided perception, for mple, is the underlying fluidity of matter, the 
indeterminate dance of electrons, "snap, crackle, and pop" of quanta, the 
tunneling and transpositions, nonlo-and superluminal, that the new physics 
presents. It is these patterns of events, >e new exhilarating metaphors of 
existence — nonlinear, uncertain, layered, discontinuous — that the computer 
can redescribe. With the computer, and 
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brought together in the telematic embrace, we can hope to glimpse the unsee 
to grasp the inehable chaos of becoming, the secret order of disorder. And a 

come to see more, we shall see the computer less and less. It will become in 

ble in its immanence, but its presence will be palpable to the artist engaged te 
atically in the world process of autopoiesis, planetary self-creation. 

The technology of computerized media and telematic systems is no long be 
viewed simply as a set of rather complicated tools extending the ranj 
painting and sculpture, performed music, or published literature. It can no 
seen to support a whole new field of creative endeavor that is as radically u 
each of those established artistic genres as they are unlike each other. A nei 
hiele of consciousness, of creativity and expression, has enterad our repertd 
being. While it is concerned with both technology and poetry, the virtual an 
immaterial as well as the palpable and concrete, the telematic may be cal 
rized as neither art nor science, while being allied in many ways to the discoi of 
both. The further development of this field will clearly mean an interde dence 
of artistic, scientific, and technological competencies and aspirations urgently, 
on the formulation of a transdisciplinary education. 

So, to link the ancient image-making process of Navajo sand painting t digital 
imaging of modern supercomputers through common silicon, W serves them 
both as pigment and processor chip, is more than ironic whimsy, holistic 
ambition of Native American culture is paralleled by the holistic p* tiality of 
telematic art. More than a technological expedient for the interch of 
information, networking provides the very infrastructure for spiritual i change 
that could lead to the harmonization and creative development o whole 
planet. With this prospectus, however na'ively optimistic and traru dental it 
may appear in our current fin-de-siecle gloom, the metaphor of lo the telematic 
embrace may not be entirely misplaced. 



Kit balloway and Sherne Rabmowitz 

"Welcome to 'Electronic Cafe 
International': A Nice Place For Hot 
CoFFee, leed Tea, £7 Virtual Space" 
(1992) 
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Dancers Keija, Mitsuko, and Soto 3000 miles apart! Satellite Arts 

Project: ASpace with No Geographical Boundaries. © 7977 
Ganoway/Rabinowitz. 

" We first explored composite-image space because we wanted to 
create a performance place with no geographical boundaries, in 
which we cou/d collaborate with other artists. " 
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« In 1984, Kit Galloway and Sherrie Rabinowitz debuted the prototype of 
Electronic Cafe by linking five cafes in various communities jn Los Angeles to 
Museum of Contemporary Art for the Olympic Arts Festival. Five years later t 
opened the Electronic Cafe International jn Santa Monica, California, wlj they 
described as a "cafe for the global village," invoking the spirit of MarsJ McLuhan. 
ECI could be considered the first cybercafe, but with a crucial difl ence from 
today's cybercafes. While it served food, drinks, and teleconferenci ECI was 
primarily a cultural research lab. When few others had yet to see need, Galloway 
and Rabinowitz brought real-time networked collaborative n timedia 
environments to people without other access. ECl's projects addres multicultural 
communities and artists across the globe. By making cutting-e 
telecommunications tools available in a casual, artists'cafe space (rather ti on a 
campus or at a museum), ECI was the catalyst for much formal innoval in 
collaborative networked performance. 

For Galloway and Rabinowitz, the cafe was the culmination of projects t dated 
to 1975, when they began their "Aesthetic Research jn Telecommurt tions" — as 
they referred to their groundbreaking satellite art at the time, spired by the 
telematic experiments of Myron Krueger, they invited artisu col laborate in "a 
performance space with no geographic boundaries." In tlj live performances, the 
images of geographically dispersed artists were tr A mitted via satellite to create a 
composite performance video on a single sen The objective, as they explained, 
was "to demonstrate (for the first time), t several performing artists, all of whom 
would be separated by oceans and ge raphy, could appear and perform together 
in the same live image (the imag< place)."These investigations jn 
"teleperformance" opened up new possibilities simultaneous theater events 
involving dancers, actors, and musicians networ across great distances. 

Galloway and Rabinowitz's investigations with artists led them to expio variety 
of unscripted forms of communication using these tools. Perhaps ti best-known 
work in this vein is the 1980s Hole-in-Space, a two-way satellite hoo between 
Lincoln Center in New York and Century City in Los Angeles. La screens were set 
up outside at both locations, projecting the image of the en that gathered at the 
opposite place, bringing people three thousand miles aj face-to-face in real time. 
In effect, each screen became a window onto the of location. Over three days, 
the project escalated into a frenzy of homecomii long lost siblings greeting one 
another, and transcontinental romances. Ha Space sparked an interest in new 
genres of telecommunications art for the In net, including MUDs, Net art, and 
networked installations. » 
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original idea, back in the 1970s, was to create an electronic, multimedia 
mposite-image space in which people separated by distance, language, 
values, d culture could come together to collaborate. The image itself would 
become ilace that we named "virtual space," which differs from virtual and 
artificial re-ty because the people, places, and things are real-world Items; 
they're not ren-red by computen Our original research was Minded by 
Corporation for Public oadcasting and National Endowment for the Arts, with 
support from NASA. We wanted performing artists to meet in composite- 
image space that mixed 2 images from remote places and presented the mix 
at each location so per-mers could see themselves on the same screen with 
their partners. That be-ne the premise for our subsequent work and 
experimentacion. This sort of composite image space was explored in the 
early 1950s by Ernie vacs and Steve Alien. Kovacs would superimpose his 
live video image onto i clips and interact with the film characters. Alien would 
do improvisational nedy by superimposing his live studio image with live 
camera shots of un-pecting pedestrians outside the television studio. 
We first explored composite-image space because we wanted to create a per- 
nance place with no geographical boundaries, in which we could collaborate i 
other artists. We wanted to explore the aesthetics and sense of presence in 
lared performance/multimedia environment, where people don't leave their 
igenous environments. That way people from varied creative and cultural 
kgrounds could help create a new environment in which they would collab-e 
on an international scale. 

tn designing such spaces, we look not only at their qualities and aesthetics, 
how people interact when they are disembodied and their image is their 
bassador." A virtual space creates social situations without tradicional rules 
iquette. The absence of the threat of physical harm makes people braver. Vir- 
space diminishes our fears of interaction. Still, a person is offended when the 
al-space hand "touches" body parts that wouldn't be touched normally "in 

lesh." In virtual space, we learn the extent to which we "own" our image. It 
;s to the beliefs of some cultures; if you take a photograph of people, they be- 
you capture or "steal" part of their essence or soul. This is a violacion of 
i image. Anyone can experience this in virtual space when seeing their image 
ted by a person or object that occupies the same image space, 
you define the aesthetic of the medium by defining the essence and in- 
y of the medium, then the creation of "good" art, in the case of telecom- 
cations, means you create a situation that provides 
some ibrm of 

uuiiicadon between people and maximizes the technology's capabilities. 
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But there must be a quality of tension that defines the "communication." Ifv 
don't create tension in the work, you're not really looking at the qualities of I 
medium, or the qualities t>f the art.... 

Hole-in-Space, from 1980, was a live two-way satellite connection using vid 
screens to project life-size images. Without public announcement, we install 
Hole-in-Space on the street near New York City's Lincoln Center and in a disp 
window at a Los Angeles department store in Century City. Both screens coi 
accommodate the images of about 15 people. People could converse with p< 
pie in the other city as if they were standing on the same corner. Hole-in-Spt 
took place over three evenings. On the first night, people simply discovered 
They went home arid told their friends, who showed up for the second evenii 
when it was also announced on TV in both locations. On the third night, fai 
lies from miles away drove into the cities with cars full of kids to meet up with r 
atives on the other coast whom they hadn't seen in years.... 



the cafe as cultural community center 

We wanted to create an environment for creative people to enjoy one-to-one a 
group-to-group exchanges. That's where Electronic Cafe comes in. The flj 
ship cafe is in southern California, in Santa Monica, networked with Electroi 
Cafe Affiliates in San Francisco, New York City, Oakland and Santa Cruz, Cs 
fornia. Santa Fe, Chicago, Pittsburgh, Vancouver, Toronto, Paris, Berlin, Japj 
Seoul. Managua, Barcelona, and Budapest. We're building human networks 
an international, cross-cultural, multidisciplinary scale. These networks cona of 
people who are getting to know each other and the technology that maintai 
their relationship. They're trymg to creatively animate that technology by < 
gaging in new types of conversation. 

The point of Electronic Cafe is to create a community commons. The 
mosphere encourages friendly conversation and serves as an amenity to I 
telecommunications technology. And our events typically cost $3-$5 to alten 

In 1984 we were commissioned by the L.A. Museum of Contemporary / 
(MOCA) to create a seven-week project for the L.A. Olympic Arts Festival. Tl 
was the birth of the Electronic Cafe concept. We linked MOCA with five divei 
L.A. communities through a telecom system, computer database, and dial-i 
image bank. People in the Korean community, Hispanic community, black coi 
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munity, and artsy beach community could send each other slow-scan video 
images, draw or write together with an electronic writing tablet, print pictures 
with the video printer, enter and retrieve information and ideas in the computer 
database, and store or retrieve images on a videodisc recorder that held 
20,000 images. Electronic Cafe Artists-in-Residence helped people use the 
systems, solicit performance pieces, and transmit musical events. 
These events propelled the network into a dynamic, inspirational existence. 
For example, Los Angeles's Korean community and black community tradition- 
ally are at odds with each other. As they met and got to know each other 
through the network, they wanted to visit each other! When you can make eye 
contad and establish a creative relationship with someone, you usually want 
to meet and touch them. At first people talked informally over the network, 
then created something together, and in many cases, ended up arranging in- 
person meetings. Today, each Electronic Cafe contains the low-cost 
Commodore Amiga com-puters, which have audio and video capabilities. We 
also use black-and-white, still-frame videophones (we cali them "vidphones"); 
color still-frame vidphones; video projectors; IBM PC-compatible and 
Macintosh II computers containing special circuit boards that capture video 
images; modems; computer printers; fax machines; and software for 
telecommunications, desktop publishing, database management, and 
graphics. We also have multiple telephone lines. Not all of the Electronic Cafes 
own video and color printers, but some locations consider them essential for 
documenting events. 

All of this hardware and software lets EC Network affiliates exchange still- 
frame video images and computer graphics, enjoy video and audio 
teleconfer-encing and two-way drawing and writing in shared-screen 
presentations, exchange computer files, programs, electronic mail, and black- 
and-white images and/or text, and conduct real-time "chats" via computer. 
People who own videophones or computers are welcome to dial in and join the 
network. We invite people to come in and encounter the technology in a 
nonintimidating environment. Our Robot Research 1200C video transmission 
system sends color or black-md-white images over phone lines (a color image 
measuring 256 X 256 pixels akes 36 seconds, while a black-and-white image 
of the same resolution sends in 14 seconds). Images are received by 
another Robot 1200C or Commodore A miga with special software. Our 
vidphones transmit black-and-white images vith an average 100 X 100 pixel 
resolution in seven or eight seconds. These vid-ihones (by Panasonic and 
Mitsubishi) no longer are manufactured, but other ompanies build similar 
systems. 

One typical ECI event was Earth Day Global Link '90, the only truly inter- 
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active global event to occur on Earth Day. It involved connections and telepe 
formances with Moscow, Nicaragua, Berlin, and Japan. In the connection wil 
Moscow, people discussed environmental problems and artists performed t< 
gether. Children in L.A. asked the Russians, "Do you have boles in your ozorn 
too?" 

In another Electronic Cafe presentation, a dancer named Dawn Stoppiell 
used technology to generate music on a Macintosh-based MIDI system and trij 
ger vidphones that sent her image to a New York City site. Working with Dawi 
computer musician Mark Coniglio of the Center for Experiments in Art, Infoi 
mation, and Technology (based at California Institute of the Arts) developed 
Macintosh II program, MIDI Dancer. Using MIDI Dancer and wearing wirela 
motion-monitoring devices on her arms and around her waist, Dawn transmi 
ted data about her movements to the Mac. Dawn's motions controlled the pn 
viously scored music, synchronizing it to her cadenee, controlled the lighting an 
microphone leveis, and determined when her image was captured and 
transmi ted to the New York location. 

In a current Electronic Cafe project, The 21st Century Odyssey, performam 
artist Barbara T. Smith is traveling around the world with a portable vidphou 
system to connect with Electronic Cafe and the artists and other people sh 
meets in her journeys. Her virtual spaces also include Bio Sphere 2 resident D 
Roy Walford, communicating with us and the world through his ECI vidphon 
system. 

Today ECI is testing limited-motion video systems and groupware. We ai 
exploring ways in which digital networks allow group-to-group activities an 
enable a person to control objects in remote rooms. 



integration and access 

If there is one word that defines Electronic Cafe, it is integration: integration ( 
technology into our social fabric; integration of distinct cultures and commun 
ties, the arts, and the general public; and integration of art forms. 
When we put integration capabilities in a community center where peopl can 
witness the "creative animation" of technology, we can start to liberate pet pie's 
imaginations. When artists explore this technology, we start pushing the Un its 
of technology. We create a culture that defines how the technology can b 
used and encourages cross-cultural collaborations, problem-solving, an 
decision-making. Our hope is to create a cultural environment that helps peop] 
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around the world articulate what they want in their future and determine how to 
§et there from here. 

The ability to pick up a telephone and talk to a person is the most powerful 
magic created by contemporary society. It is the only mechanism that lets us 
manage a system as large as a planet. We must recognize that this is sacred, 
and Decome a "tele-species." Through Electronic Cafes and the use of 
telecommu-lications technology we hope to foster new kinds of artistic 
collaboration and ex-jerimentation that will support the tele-species. 

We want to make teleconferencing systems nonintimidating and accessible 
to jeople who don't work within corporations. We want to get the technology to 
as nany people as possible, articulate the ramifications, define the aesthetics, 
and opple political roadblocks. Corporate culture is not going to provide 
models or pplications broad enough to visualize the technology's cultural 
application. We mst acculturate it ourselves. 

If the arts are to take a role in shaping and humanizing emerging 
technolog-:al environments, individuals and arts constituencies must start to 
imagine at a mch larger scale of creativity. If you look at the aesthetic quality 
of the commu-ication and you're true to your art form and your art logic, then 
you naturally ut one foot in front of the other ... and the art logic marches 
you right out of le art institutions and into life.. 



Pavel Curtis 

35» "Mudding: Social 

Phenomena in Text- 
Based Virtual Realities" 
(1992) 
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Pavel Curtis. Photo by Anne Hamersky, ©Anne Hamersky, 
Fast Company Magazine. 



"The emergence of MUDs has created a new kind of social sphere, 
both like and radically unlike the environments that have existed 
before. As they become more and more popular and more widely 
accessible, jt appears likely that an increasingly significant 
proportion ofthe population will at least become familiar with 
mudding and perhaps become frequent partidpants in text-based 
virtual realities. " 
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Pavel Curtis, a computar scientistat Xerox PARC (Palo Alto Research Cen-:er), 
created one of the first popular on-line role-playing environments, Lamb- 
iaMOO, in 1991. Known as a MUD (the acronym stands for Multi-User 
3ungeons), LambdaMOO is a text-only fantasy realm that is descended from ole- 
playing games from the 1970s such as "Dungeons and Dragons" (henee the 
lungeon allusion in MUD). After presiding over this experimental digital envi- 
onment for one year as its creator, administrator, and self-proclaimed "wiz-ird," 
Curtis documented his observations of the evolving social, cultural, and 
'sychological effects of interaction in cyberspace. While not the first of its kind, 
.ambdaMOO is perhaps the most famous text-based virtual environment, dis- 
ected and analyzed by media theorists, sociologists, and psychologists who see : 
as fertile breeding ground for a new hybrid form of literature, live perfor- lance, 
cinema, and interactive story making. 

In his essay, Curtis directs our attention to LambdaMOO's narrative char- 
;tenstjcs. Unlike conventional story structures, MUDs do not rely on defined 
ots with clear objectives. Rather, through the freewheeling dynamics of impro- 
sed dialogue and unrehearsed interactivity, participants lose themselves in their 
les and collaborate in a form of collective authorship. Shielded (and even lib- 
ated) by the anonymity of their characters, players improvise their own con- 
rsations, story lines, props, and settings; they pursue their own adventures, and 
periment with a myriad of alternate identities; sometimes they even switch gen- 
r and, occasionally, species. LambdaMOO has served as a model for thousands 
MUDs that have since sprung from the Internet and the World Wide Web. 
U Ds are characterized by a tightly knit — though globally dispersed — commu- 
y of characters engaged in an ongoing dialogue that combines the aimlessness 
nomadic wandering with the focused creativity of world building.There is no 
niscient narrator, no central author, in LambdaMOO, despite the shadowy 
sence of its "wizard," 
Pavel Curtis. 

While Curtis emphasizes the richness of the text-based virtual world — re- 
-ing to Marshall McLuhan's concept of the "cool" medium in which the 
wer's imagination is engaged to A complete" the experience of the low- 
ilution environment — graphic, multimedia virtual worlds have supplanted 
r textual ancestors. Nonetheless, Curtis's observation of on-line role playing 
virtual communities are fundamental to our understanding of the social im- 
ations of networked alternative realities, now popular in the form of chat 
ns, teleconferencing, and email Listservs. » 
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around the world articulate what they want in their future and determine how to 
get there from here. 

The ability to pick up a telephone and talk to a person is the most powerful 
magic created by contemporary society. It is the only mechanism that lets us 
manage a system as large as a planet. We must recognize that this is sacred, 
and become a "tele-species." Through Electronic Cafes and the use of 
telecommu-nications technology we hope to foster new kinds of artistic 
collaboration and ex-perimentation that will support the tele-species. 

We want to make teleconferencing systems nonintimidating and accessible 
to people who don't work within corporations. We want to get the technology to 
as many people as possible, articulate the ramifications, define the aesthetics, 
and topple political roadblocks. Corporate culture is not going to provide 
models or applications broad enough to visualize the technology's cultural 
application. We must acculturate it ourselves. 

If the arts are to take a role in shaping and humanizing emerging technolog- 
ical environments, individuals and arts constituencies must start to imagine at 
a much larger scale of creativity. If you look at the aesthetic quality of the 
commu-nication and you're trae to your art form and your art logic, then you 
naturally put one foot in front of the other ... and the art logic marches you 
right out of the art institutions and into life.. 
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retched across the hall. There are doorways leading 
the north and south. 

ju see a sign hanging from the middle of the rope here, 
read sign 

lis point marks the end of the currently-occupied 
Drtion of the house. Guests proceed beyond this point 
their own risk. 
the residents 
jo east 

su step disdainfully over the velvet rope and enter 
le dusty darkness of the unused portion of the house. 



Three major factors distinguish a MUD from an Adventure-style computer 
game, lough: 

3. A MUD is not goal-oriented; it has no 
beginning or end, no "score," andno 
notion of "winning" or "success." In short, 
even though users of MUDsare 
commonly called players, a MUD isn't 
really a game at all. 

4. A MUD is extensible from within; a 
user can add new objects to the data 
base such as rooms, exits, "things," 
and notes. Certain MUDs, including 
the one I run, even support an 
embedded programming language in 
whicha user can describe whole new 
kinds of behavior for the objects they 
create. 



5. A MUD generally has more than 
one user connected at a time. All of 
theconnected users are browsing 
and manipulating the same database 
andean encounter the new objects 
created by others. The multiple users 
on aMUD can communicate with 
each other in real time. 



is last factor has a profound effect on 
the ways in which users interact with 
the tern; it transforms the activity from 
a solitary one into a social one. Most 
inter-player communication on MUDs 
follows rules that fit within the nework 
of the virtual reality. If a player "says" 
something (using the say corn-id), then 
every other player in the same room 
will "hear" them. For example, pose 
that a player named Munchkin typed 
the command 



say Can anyone hear me? 

In Munchkin would see the feedback 

¥ou say, "Can anyone hear me?" 

every other player in the same room would see 

klunchkm says, "Can anyone hear me?" 
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Similarly, the emote command allows players to express various forms of "non 
verbal" communication. If Munchkin types 

emote smiles. 

then every player in the same room sees Munchkin smiles. 

Most interplayer communication relies entirely on these two commands. 3 

There are two circumstances in which the realistic limitations of say ant 
emote have proved sufficiently annoying that new mechanisms were 
developed It sometimes happens that one player wishes to speak to another 
player in th< same room, but without anyone else in the room being aware of 
the communication. If Munchkin uses the whisper command 

whisper "I wish he'd just go away ..." to Frebble then only Frebble will see 

Munchkin whispers, "I wish he'd just go away ..." 

The other players in the room see nothing of this at all. 

Finally, if one player wishes to say something to another who is connected 
to the MUD but currently in a difFerent and perhaps "remote" room, the page 
command is appropriate. It is invoked with a syntax very like that of the 
whisper command and the recipient sees output like this: 

You sense that Munchkin is looking for you in The Hall. He pages, "Come see 
this clock, it's tres cool!" 

Aside from conversation, MUD players can most directly express 
themselves in three ways: by their choice of player name, by their choice of 
gender, and by their self-description. 

When a player first connects to a MUD, they choose a name by which the 
other players will know them. This choice, like almost all others in MUDs, is not 
cast in stone; any player can rename themself at any time, though not to a 
name currently in use by some other player. Typically, MUD names are single 
words, in contrast to the longer "full" names used in real life. 

Initially, MUD players appear to be neuter; automatically generated 
messages that refer to such a player use the family of pronouns including "it," 
"its," etc. Players can choose to appear as a difFerent gender, though, and not 
only male or female. On many MUDs, players can also choose to be plural 
(appearing to be a 
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tind of "colony" creature: "ChupChups leave the room, closing the door 
behind them"), or to use one of several sets of gender-neutral pronouns (e.g., 
"s/he," ''him/her" and "his/her," or "e," "em" and "eir"). 

Every object in a MUD optionally has a textual description which players can 
view with the look command. For example, the description of a room is auto- 
matically shown to a player when they enter that room and can be seen again 
just by typing "look." To see another player's description, one might type 
"look Bert." Players can set or change their descriptions at any time. The 
lengths of player descriptions typically vary from short one-liners to dozen-line 
paragraphs. 

Aside from direct communication and responses to player commands, 
mes-sages are printed to players when other players enter or leave the same 
room, when others connect or disconnect and are already in the same room, 
and when objects in the virtual reality have asynchronous behavior (e.g., a 
cuckoo clock chiming the hours). 

MUD players typically spend their connected time sociali/ing with each 
other, exploring the various rooms and other objects in the database, and 
adding new such objects of their own design. They vary widely in the amount 
of time they spend connected on each visit, ranging from only a minute to 
several hours; some players stay connected (and almost always idle) for days 
at a time, only oc-casionally actively participating. 

This very brief description of the technical aspects of mudding suffices for the 
purposes of this paper. It has been my experience, however, that it is quite 
diffi-cult to properly convey the "sense" of the experience in words. Readers 
desiring more detailed information are advised to try mudding themselves, as 
described in the final section of this paper. 
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2 social phenomena 
observed on one 
MUD 



Man is the measure. 4 



— E. M. 

Forster 
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In October of 1990,1 began running an Internet-accessible MUD server on 
my personal workstation here at PARC. Since then, it has been running 
continuously, with interruptions of only a few hours at most. Injanuary of 1991, 
the existence of the MUD (called LambdaMOO 5 ) was announced publicly, 
via the Usenet newsgroup rec.games.mud. As of this writing, well over 3,500 
difieren! players 
i\>a raveiiums 



have connected to the server from over a dozen countries around the world 
and, at any given time, over 750 players have connected at least once in the 
last week. Recent statistics concerning the number of players connected at a 
given time of day (Pacific Standard Time) appear below. 

42. a.m. ************** 10-1/2 

43. a.m. ***************** 12/1/4 
44 a m ******************* ]_4 

I 3 In ************************** 

21-1/4 

23 ^ m ****************************** 

21-1/4* 

29. a.m. 

*********************************** 2 5 — 1/4 



11. 


a.m. 


* * * * 


******** 


12. 


a.m. 


* * * * 


******** 


* * * * 


** 32-1/ 


* * * * 


******** 


* * * * 


******** 



11. p.m. 

******************************************** 
************** 41 — 1/4 

12. p.m. 

******************************************* 
************* 39—3/4 

13. p.m. 

******************************************* 

****** -j Cj 

14. p.m. 

******************************************* 
************* 3 g 1/2 

15. p.m. 

******************************************** 
************** 40—3/4 

16. p.m. 



************* 39— 3/4 

17. p.m. 

******************************************* 
************** 40—1/2 

18. p.m. 

******************************************* 
***************** 

42-1/2 

9p.m. 

******************************** 
******************* 

44-1/4 

12. p.m. 

******************************************* 
********* 37-3/4 

13. p.m. 

********************************* 
************ 31 

mi dn icrh t ************************ 
*************** 26 _ 3/4 

1. a.m. 

**************************************** 
20-3/4 

2 am ******************* 13-3/4 

3 am ************** 10—3/4 
4. a.m. ************* 1Q-1/2 



Lambda MOO is clearly a reasonably active place, with new and oid players 
com-ing and going frequently throughout the day. This popularity has 
provided me with a position from which to observe the social patterns of a fairly 
large and diverse MUD dientele. I want to point out to the reader, however, that I 
have no formal training in sociology, anthropology, or psychology, so I cannot 
make any claims about methodology or even my own objectivity. What I relate 
below ia merely my personal observations made over a year of mudding. In 
most cases, my discussions of the motivations and feelings of individual players 
is based upon in-MUD conversations with them; I have no means of checking 
the veracity of their statements concerning their real-life genders, identities, or 
(obviously) feelings. On the other hand, in most cases, I also have no reason 
to doubt them. 

I have grouped my observations into three categories: phenomena related 
to the behavior and motivations of individual players, phenomena related to 
inter- 
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actions between small groups of players (especially observations concerning 
MUD conversation), and phenomena related to the behavior of a MUD's 
com-munity as a whole. 

Cutting across all of these categories is a recurring theme to which I would 
like to draw the reader's attention in advance. Social behavior on MUDs is in 
some ways a direct mirror of behavior in real life, with mechanisms being 
drawn nearly unchanged from real-life, and in some ways very new and 
different, taking root in the new opportunities that MUDs provide over real life. 

2. 1 Observations about individuals 

The Mudding Population The people who have an opportunity to connect to 
LambdaMOO are not a representative sample of the world population; they 
all read and write English with at least passable proficiency and they have 
access to the Internet. Based on the names of their network hosts, I believe 
that well over 90% of them are affiliated with colleges and universities, mostly 
as students and, to a lesser extent, mostly undergraduates. Because they 
have Internet access, it might be supposed that the vast majority of players 
are involved in the comput-ing field, but I do not believe that this is the case. It 
appears to me that no more than half (and probably less) of them are so 
employed; the increasing general availability of computing resources on 
college campusas and in industry appears to be having an effect, allowing a 
broader community to participate. 

In any case, it appears that the educational background of the mudding 
community is generally above average and it is likely that the economic 
background is similarly above the norm. Based on my conversations with 
people and on the names of those who have asked to join a mailing list about 
programming in LambdaMOO, I would guess that over 70% of the players are 
male; it is very difficult to give any firm justification for this number, however. 



Player Presentation As described in the introduction to mudding, players 
have a number of choices about how to present themselves in the MUD; the 
first such decision is the name they will use. Some of the names used by 
players on LambdaMOO are shown below. 



Gemba 


GarySevern 


Ford 


Frand 


Maya 


Rincewind 


yduj 


funky 


Foodslave 


Arthur 


EbbTide 


Anathae 


Satan 


byte 


Booga 


tek 



Toon li'ir 
Grump yrx 

Pavel Curtis 



chupchups 


waffle 


Miranda Gus 


Merlin 


Moonlight 


MrNatural 


Winger 


Drazz'zt 


Kendal 


Redjack 


Snooze 


Shin 


lostboy 


foobar 


Ted Logan 


Xephyr 


King_Claudius 


Bruce 


Puff 


Dirque 


Coyote 


Vastin 


Player 


Cool 


Amy 


Thorgeir 


Cyberhuman 


Gandalf 


blip 


Jayhirazan 


Firefoot 


JoeFeedback 


ZZZzzz . . . 


Lyssa 


Avalar 


zipo 


Blackwinter 


viz 


Kilik 


Maelstorm 


Love 


Terry arm 


Chrystal 


arkanoiv 



One can pick out a few common styles for names (e.g., names from or ir 
spired by myth, fantasy, or other literature, common names from real life, name 
of concepts, animais, and everyday objects that have representative 
connotations etc.), but it is clear that no such category includes a majority of 
the names. Not that a significant minority of the names are in lower case; this 
appears to be a styl istic choice (players with such names describe the practice 
as "cool") and not, a might be supposed, an indication of a depressed ego. 

Players can be quite possessive about their names, resenting others wh( 
choose names that are similarly spelt or pronounced or even that are taken 
fron the same mythology or work of literature. In one case, for example, a 
playe: names "ZigZag" complained to me about other players taking the 
name; "ZigZag!"and"Zig." 

The choice of a player's gender is, for some, one of great consequence 
anc forethought; for others (mosdy males), it is simple and without any 
questions. Foi all that diis choice involves the fewest options for the player 
(unlike their name or description, which are limited only by their imagination), it 
is also the choice that can generate the greatest concern and interest on the 
part of other players. 



As I've said before, it appears that the great majority of players are male 
and the vast majority of them choose to present themselves as such. Some 
males, however, taking advantage of the relative rarity of females in MUDs, 
present themselves as female and thus stand out to some degree. Some use 
this distinc-tionjust for the fun of deceiving others, some of these going so far 
as to try to entice male-presenting players into sexually explicit discussions 
and interactions. This is such a widely noticed phenomenon, in fact, that one 
is advised by the common wisdom to assume that any flirtatious female- 
presenting players are, in real life, males. Such players are often subject to 
ostracism based on this as-sumption. 

Some MUD players have suggested to me that such transvestite flirts are 

per- 
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s acting out their own (latent or otherwise) homosexual urges or fantasies, tak- 
advantage of the perfect safety of the MUD situation to see how it feels to ap- 
ach other men. While I have had no personal experience talking to such 
rers, let alone the opportunity to delve into their motivations, the idea strikes as 
plausible given the other ways in which MUD anonymity seems to free peo-from 
their inhibitions. (I say more about anonymity later on.) Other males present 
themselves as female more out of curiosity than as an at->pt at deception; to 
some degree, they are interested in seeing "how the other :"lives," what it feels 
like to be perceived as female in a community. From what n tell, they can be 
quite successful at this. 

Female-presenting players report a number of problems. Many of them have 1 
me that they are frequently subject both to harassment and to special treat- it. 
One reported seeing two newcomers arrive at the same time, one male- 
senting and one female-presenting. The other players in the room struck up 
versations with the putative female and offered to show her around but com- 
ely ignored the putative male, who was left to his own devices. In addition, 
probably due mostly to the number of fernale-presenting males hears about, 
many female players report that they are frequently (and some-is quite 
aggressively) challenged to "prove" that they are, in fact, female. To best of my 
knowledge, male-presenting players are rarely if ever so chal-;ed. 
Because of these problems, many players who are female in real life choose 
resent themselves otherwise, choosing either male, neuter, or gender-neutral 
louns. As one might expect, the neuter and gender-neutral presenters are still 
ect to demands that they divulge their real gender. 

Some players apparently find it quite difficult to interact with those whose 
gender has been called into question; since this phenomenon is rarely man-in 
real life, they have grown dependen! on "knowing where they stand," on ving 
what gender roles are "appropriate." Some players (and not only males) feel 
that it is dishonest to present oneself as being a different gender than in life; 
they report feeling "mad" and "used" when they discover the deception. Vhile I 
can spare no more space for this topic, I enthusiastically encourage 



nterested reader to look up Van Gelder's fascinating article 6 for many more 
iples and insights, as well as the story of a remarkably successful deception 
electronic transvestism." 



The final part of a player's self-presentation, and the only part involving e, 
IS the player's description. This is where players can, and often do, es-sh 
the details of a persona or role they wish to play in the virtual reality. It is 
nlso a significant factor in other players' first impressions, since new players ai 
commonly looked at soon after entering a common room. 

Some players use extremely short descriptions, either intending to be cry] 
tic (e.g., "the possessor of the infinity gems") or straightforward (e.g., " A 
average-sized dark elf with lavender eyes") or, often, [they are] just insufHciend 
motivated to create a more complex description for themselves. Other players 
g to great efforts in writing their descriptions; one moderately long example 
ap pears below. 
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You see a quiet, unassuming figure, wreathed in an oversized, dull-green Army 
jacket which is pulled up to nearly conceal his face. His long, unkempt blond hair 
blows back from his face as he tosses his head to meet your gaze. Small round 
gold-rimmed glasses, tinted slightly grey, rest on his nose. On a shoulder strap 
he carnes an acoustic guitar and he lugs a backpack stuffed to overflowing with 
sheet music, sketches, and computer printouts. Under the coat are faded jeans 
and a T-shirt reading "Paranoid CyberPunks International." He meets your gaze 
and smiles faintly, but does not speak with you. As you surmise him, you notice a 
glint of red at the rims of his blue eyes, and realize that his canine teeth seem to 
protrude slightly. He recoils from your look of horror and recedes back into 
himself. 

A large proportion of player descriptions contain a degree of wish fulfil] 
ment; I cannot count the number of "mysterious but unmistakably powerful" fig 
ures I have seen wandering around in LambdaMOO. Many players, it seems, 
ah taking advantage of the MUD to emulate various attractive characters from 
fie tion. 

Given the detail and conteni of so many player descriptions, one might expec 
to find a significant amount of role-playing, players who adopt a coherent char 
acter with features distinct from their real-life personalities. Such is rarely th< 
case, however. Most players appear to tire of such an effort quickly and simply in 
teract with the odiers more-or-less straightforwardly, at least to the degree on< 
does in normal discourse. One factor might be that the roles chosen by 
playen are usually taken from a particular creative work and are not 
particularly viabh as characters outside of the context of that work; in short, 
the roles don't mala sense in the context of the MUD. 

A notable exception to this rule is one particular MUD Tve heard of, callee 
"PernMUSH." This appears to be a rigidly maintained simulacrum of the worlc 
described in Ann McCaffrey's celebrated "Dragon" books. All players there hav« 
names that fit the style of the books and all places built there are consistent 
witt 
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what is shown in the series and in various fan materials devoted to it. 
PernMUSH apparendy holds frequent "hatchings" and other social events, 
also derived in great detail from McCaffrey's works. This exception probably 
succeeds only be-cause of its single-mindedness; with every player providing 
the correct context for every other, it is easier for everyone to stay more-or- 
less "in character." 

Player Anonymity It seems to me that the most significant social factor in 
MUDs is the perfect anonymity provided to the players. There are no 
commands available to the players to discover the real-life identity of each 
other and, in-deed, technical considerations make such commands either very 
difficult or im-possible to implement. 

It is this guarantee of privacy that makes players' self-presentation so 
impor-tant and, in a sense, successful. Players can only be known by what 
they explic-idy project and are not "locked into" any factors beyond their easy 
control, such as personal appearance, race, etc. In the words of an oid military 
recruiting com-mercial, MUD players can "be all that you can be." 7 

This also contributes to what might be called a "shipboard syndrome," the 
feeling that since one will likely never meet anyone from the MUD in real life, 
there is less social risk involved and inhibitions can safely be lowered. 

For example, many players report that they are much more willing to strike 
up conversations with strangers they encounter in the MUD than in real life. 
One obvious factor is that MUD visitors are implicitly assumed to be interested 
in conversing, unlike in most real world contexts. Another deeper reason, 
though, is that players do not feel that very much is at risk. At worst, if they 
feel diat they've made an utter fool of themself, they can always abandon die 
character and create a new one, losing only the name and the effort invested 
in socially estab-lishing the oid one. In effect, a "new lease on life" is always a 
ready option. 

Players on most MUDs are also emboldened somewhat by the fact that 
diey ire immune from violence, both physical and virtual. The permissions 
systems of all MUDs (excepting those whose whole purpose revolves around 
adventur-ing and the slaying of monsters and other players) generally 
prevent any player From having any kind of permanent effect on any other 
player. Players can cer-:ainly annoy each other, but not in any lasting or 
even moderately long-lived nanner. 

This protective anonymity also encourages some players to behave irre- 
iponsibly, rudely, or even obnoxiously. We have had instances of severe and 
re->eated sexual harassment, crudity, and deliberate offensiveness. In 



general, such :ruelty seems to be supported by two causes: the offenders 
believe (usually cor- 
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reedy) that they cannot be held accountable for their actions in the real world, 
and the very same anonymity makes it easier for them to treat other players 
imper-sonally, as other than real people. 

Wizards Usually, as I understand it, societies cope with offensive behavior by 
various group mechanisms, such as ostracism, and I discuss this kind of effect 
in detail in Section 2.3. In certain severe cases, however, it is left to the 
"authorities" or "police" of a society to take direct action, and MUDs are no 
different in this respect. 

On MUDs, it is a special class of players, usually called wizards or (less fre- 
quently) gods, who fulfill both the "authority" and "police" roles. A wizard is a 
player who has special permissions and commands available, usually for the 
pur-pose of maintaining the MUD, much like a "system administrator" or 
"super-user" in real-life computing systems. Players can only be transformed 
into wizardsby other wizards, with the maintainer of the actual MUD server 
computer pro-gram acting as the first such 

2.2 Observations about small groups 

MUD Conversation The majority of players spend the majority of their active 
time on MUDs in conversation with other players. The mechanisms by which 
those conversations get started generally mirror those that operate in real 
life, though sometimes in interesting ways. 

Chance encounters between players exploring the same parts of the 
database are common and almost always cause for conversation. As 
mentioned above, the anonymity of MUDs tends to lower social barriers and to 
encourage players to be more outgoing than in real life. Strangers on MUDs 
greet each other with the same kinds of questions as in real life: "Are you new 
here? I don't think we've met." The very first greetings, however, are usually 
gestural rather than verbal: "Munchkin waves. Lorelei waves back." 
The @who (or WHO) command on MUDs allows players to see who else is 
currently connected and, on some MUDs, where those people are. An 
example of the output of this command appears below. 



PLAYER NAME CONNECTED IDLETIME LOCATION 
Haakon (#2) Lynx(#8910) Garin (#23393) 
3 days a minute an hour 
a second 2 seconds 2 seconds 
Lambda's Den Lynx' Abode Carnival Grounds 
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PLAYERNAME CONNECTED IDLETIME 



LOCATION 



Gilmore (#19194) an 
hour 


10 

seconds 


Heart of 
Darkness 


TamLin(#21864) 
anhour 


21 

seconds 


Heart of 
Darkness 


Quimby (#23279) 3 
minutes 


2 minutes 


Quimby's room 


koosh (#24639) 50 
minutes 


5 minutes 


Corridor 


Nosredna (#2487) 7 
hours 


36 

minutes 


Nosredna's 
Hideaway 


yduj (#68) 7 
hours 


47 

minutes 


Hackers' Heaven 


Zachary (#4670) an 
hour 


an hour 


Zachary's 
Workshop 


Woodlock(#2520) 2 
hours 


2 hours 


Woodlock's 
Room 



Total: 1 1 players, 6 of whom have been active recently. 

This is, in a sense, the MUD analog of scanning the room in a real-life 
gathering to see who's present.... 

Other Small-Group Interactions I would not like to give the impression that 
conversation is the only social activity on MUDs. Indeed, MUD society appears 
to have most of the same social activities as real life, albeit often in a modified 
form. As mentioned before, PernMUSH holds large-scale, organized social 
gath-erings such as "hatchings" and they are not alone. Most MUDs have at 
one time or another organized more or less elaborate parties, often to 
celebrate notable events in the MUD itself, such as an anniversary of its 
founding. We have so far Nad only one or two such parties on LambdaMOO, 
to celebrate the "opening" Df some new area built by a player; if there were 
any other major parties, I cer-tainly wasn't invited! 

One of the more impressive examples of MUD social activity is the virtual 
vedding. There have been many of these on many different MUDs; we are in 



the Drocess of planning our first on LambdaMOO, with me officiating in my role 
as irchwizard. 

I have never been present at such a ceremony, but I have read logs of the 
jonversations at them. As I do not know any of the participants in the cere- 
nonies Tve read about, I cannot say much for certain about their emotional 
con-ent. As in real life, they are usually very happy and celebratory occasions 
with n intriguing undercurrent of serious feelings. I do not know and cannot 
even peculate about whether or not the main participants in such 
ceremonies are .sually serious or not, whether or not the MUD ceremony 
usually (or even ever) urrors another ceremony in the real world, or even 
whether or not the bride and room have ever met outside of virtual reality. 

In the specific case of the upcoming LambdaMOO wedding, the partici- 
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pants first met on Lambda MOO, became quite friendly, and eventually decidec 
to meet in real life. They have subsequently become romantically involved in th< 
real world and are using the MUD wedding as a celebration of that fact. This 
phe nomenon of couples meeting in virtual reality and then pursuing a real-life 
rei» tionship is not uncommon; in one notable case, they did this even though 
one of them lived in Australia and the other in Pittsburgh! 

It is interesting to note that the virtual reality wedding is not specific to thi 
kinds of MUDs Tve been discussing; Van Gelder 8 mentions an on-line receptior 
on CompuServe and weddings are quite common on Habitat, 9 a half-graphicai 
half-textual virtual reality popular in Japan. 

The very idea, however, brings up interesting and potentially importan! ques- 
tions about the legal standing of commitments made only in virtual reality. Sup- 
pose, for example, that two people make a contract in virtual reality. Is tht 
contract binding? Under which state's (or country's) laws? Is it a written or 
verbal contract? What constitutes proof of signature in such a context? I suspect 
thai our real-world society will have to face and resolve these issues in the 
not-too-distant future. 

Those who frequent MUDs tend also to be interested in games and 
puzzles, so it is no surprise that many real-world examples have been 
implemented inside MUDs. What may be surprising, however, is the extent to 
which this is so. 

On LambdaMOO alone, we have machine-mediated Scrabble, Monopoly, 
Mastermind, Backgammon, Ghost, Chess, Go, and Reversi boards. These 
attracl small groups of players on occasion, with the Go players being the 
most com-mitted; in fact, there are a number of Go players who come to 
LambdaMOO only for that purpose. I say more about these more specialized 
uses of social virtual re-alities later on. In many ways, though, such games so 
far have little, if anything, to offer over their real-world counterparts except 
perhaps a better chance of find-ing an opponent. 

Perhaps more interesting are the other kinds of games imported into 
MUDs from real life, the ones that might be far less feasible in a non-virtual 
reality. A player on LambdaMOO, for example, implemented a facility for 
holding food fights. Players throw food Items at each other, attempt to duck 
oncoming Items, and, if unsuccessful, are "splattered" with messes that 
cannot easily be removed. After a short interval, a semi-animate "Mr. Clean" 
arrives and one-by-one removes the messes from the participants, turning 
them back into the food Items from which they carne, ready for the next fight. 



Although the game was rather simple to implement, it has remained 
enormously popular nearly a year later. 

Another player on LambdaMOO created a trainable Frisbee, which any 
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player could teach to do tricks when they threw or caught it. Players who used 
the Frisbee seemed to take great pleasure in trying to out-do each other's trick 
de-scriptions. My catching description, for example, reads "Haakon stops the 
fris-bee dead in the air in front of himself and then daintily plucks it, like a 
flower." I Nave also heard of MUD versions of paint-ball combat and fantastical 
games of Capture the Flag. 

2.3 Observations abou'i the MUD community as a whole 

VIUD communities tend to be very large in comparison to the number of play- 
irs actually active at any given time. On LambdaMOO, for example, we have 
be-ween 700 and 800 players connecting in any week but rarely more than 
40 ^multaneously. A good real-world analog might be a bar with a large 
number of regulars," all of whom are transients without fixed schedules. 

The continuity of MUD society is thus somewhat tenuous; many pairs of ac- 
ive players exist who have never met each other. In spite of this, MUDs do be- 
ome true communities after a time. The participants slowly come to 
consensus bout a common (private) language, about appropriate standards of 
behavior, and bout the social roles of various public areas (e.g., where big 
discussions usually appen, where certain "crowds" can be found, etc.).... 
It should be noted that different MUDs are truly different communities and 
ave different societal agreements concerning appropriate behavior. There 
even dst a few MUDs where the only rule in the social contract is that there is 
no so-al contract. Such "anarchy" MUDs have appeared a few times in my 
experience id seem to be quite popular for a time before eventually fading 
away. These are the main points of LambdaMOO manners: 

3. Be polite. Avoid being rude. The MOO is worth 
participating in becauseit is a pleasant place for people to be. 
When people are rude or nasty to oneanother, it stops being so 
pleasant. 

4. "Revenge is ours," sayeth the wizards. If someone is 
nasty to you, picase ei-ther ignore it or tell a wizard about 
it. Picase don't try to take revenge on theperson; this just 
escalales the level of rudeness and makes the MOO a 
lesspleasant place for everyone involved. 



Respect other players' sensibilities. The participants on the MOO comefrom 
a wide range of cultures and backgrounds. Your ideas about what constitutes 
offensive speech or descriptions are likely to differ from thoseof other players. 
Picase keep the text that players can casually run across asfree of potentially- 
offensive material as you can. 
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Don't spoof. Spoofing is loosely defined as "causing misleading output to be 
printed to other players." For example, it would be spoofing for anyone but 
Munchkin to print out a message like "Munchkin sticks out bis tongue at 
Potrzebie." This makes it look like Munchkin is unhappy with Potrzebie even 
though that may not be the case at all. 

Don't shout. It is easy to write a MOO command that prints a message to 
every connected player. Picase don't. 

Only teleport your own things. By default, most objects (including other 
players) allow themselves to be moved freely from place to place. This fact 
makes it easier to build certain useful objects. Unfortunately, it also makes it 
easy to annoy people by moving them or their objects around without their 
permission. Picase don't. 

Don't teleport silently or obscurely. It is easy to write MOO commands that move 
you instantly from place to place. Please remember in such programs to print a 
clear, understandable message to all players in both the place you're leaving 
and the place you're going to. 

Don't hog the serven The server is carefully shared among all of the 
connected players so that everyone gets a chance to execute their 
commands. This sharing is, by necessity, somewhat approximate. Please don't 
abuse it with tasks that run for a long time without pausing. 
Don't waste object numbers. Some people, in a quest to own objects with 
"interesting" numbers (e.g., #17000, #18181, etc.) have written MOO 
programs that loop forever, creating and recycling objects until the "good r 
numbers come up. Please don't do this 



3 the prospects for 
mudding in the future 



The clumsy system of public gatherings had been long since abandonad; neither 
Vashti nor her audience stirred from their rooms. Seated in her arm-chair, she 
spoke, while they in their arm-chairs heard her, fairly well, and saw her, fairly well. 

-E. M. Forster 10 



It is substantially easier for players to give themselves vivid, 
detailed, and interesting descriptions (and to do the same for the 
descriptions and behavior of the new objects they create) in a text- 
based system than in a graphics-based one. In McLuhan's 
terminology, 11 this is because MUDs are a "cold" medium, while 
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lore graphically-based media are "hot"; that is, the sensorial parsimony of plain 
;xt tends to entice users into engaging their imaginations to fill in missing details 
'hile, comparatively speaking, the richness of stimuli in fancy virtual realities has 
n opposite tendency, pushing users' imaginations into a more passive role. I 
Iso find it dihicult to believe that a graphics-based system will be able to 
comete with text for average users on the metric of believable detail per unit of 
ef->rt expended; this is certainly the case now and I see little reason to believe it 
will tiange in the near future. 

Finally, one of the great strengths of MUDs lies in the users' ability to cus- 
imize them, to extend them, and to specialize them to the users' particular 
;eds. The case with which this can be done in MUDs is directly related to the 
ct that they are purely text-based; in a graphics-based system, the overhead 

of 

eating new moderate-quality graphics would put the task beyond the inclina- 
>ns of the average user. Whereas, with MUDs, it is easy to imagine an almost 

ar- 

trarily small community investing in the creation of a virtual reality that was 
ily customized for that community, it seems very unlikely that any but the 
•gest communities would invest the greatly-increased effort required for a 

icier system. 

CONCLUSIONS 

Vashti was seized with the terrors of direct experience. She shrank back into 
her room, and the wall closed up again. 

-E. M. Forster 12 



The emergence of MUDs has created a new kind of social sphere, both like 
and ically unlike the environments that have existed before. As they become 
more 1 more popular and more widely accessible, it appears likely that an 
increas-y significant proportion of the population will at least become 
familiar with iding and perhaps become frequent participants in text-based 
virtual reali- 

It thus behooves us to begin to try to understand these new societies, to 
make je of these electronic places where we'll be spending increasing 
amounts of time, both doing business and seeking pleasure. I would hope 
that social sci-3ts will be at least intrigued by my amateur observations 
and perhaps in-/d to more properly study MUDs and their players. In 
particular, as MUDs >me more widespread, ever more people are likely to 
be susceptible to the of addiction I discuss; we must, as a society, begin to 
wrestle with the social ethical issues brought out by such cases.... 



Pi erre Levy 

3 6» "The Art and Architecture 
oF Cyberspace," Collective 
Intelligence (1994) 



Fierre Levy. Photo by Darcia Labrosse. 
Courtesy of Fierre Levy. 



"Rather than distribute a message to 
recipients who are outside the process of 
creation and invited to give meaning to a 
work ofart belatedly, the artist now 
attempts to construct an environment, a 
system of communication and production, 
a collective event that implies its 
recipients, transforms interpreters into 
actors, enables interpretation to enter the 
loop with collective action. " 
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media theonst and philosopher Fierre Levy brings a sociological tive to the 
art and architecture of cyberspace and the emerging study of jlture. His books, 
Collective Intelligence: Mankind's Emerging World in Cyberspace and 
Becoming Virtual: ReaHtyinthe Digital Age (1997), have helped shape the 
about the aesthetic and social implications of multimedia, influencing tand 
theorists alike. A counterpoint to the dystoplc vision of Burroughs and Levy 
points to a digitally conceived Utopian universe, a virtual world in i vast 
repositories of information, decentralized authorship, mutable iden-l telematic 
interaction form an "endless horizon" of evolving forms of art lunication. 
identifies an active role for the recipient of the artwork jn tandem with tic 
dissolution of authorial control on the part of its creator. For Levy, becoming 
a dynamic, fluid, changing environment, a "deterritorialized : plane" in which 
"artist" and "recipient" unite jn a consensual interplay formation, execution, 
and interpretation of art. This shift marks a final from the notion of art as a 
sacred, discrete object, as indicated by the di-icd authority of the artist's 
signature, an indelible aspect of art since its be-gs. In its place, he views the 
digital medium as a continuous and lorative work-in-progress, 
ccording to Levy, the break from traditional notions of authorship js lead- 
toward cultural transformation. He envisions a collective society linked by riic 
networks, with citizens actively engaged in the "continuous invention i 
languages and signs of a community." Rather than approach the Internet, I 
reality, video games, and emerging media as disparate subjects, Levy iden-; 
similarities between their narrative approaches, and treats them as a uni-?ment 
toward a global culture. Levy furthers the case made by Douglas irt, and 
proposes that multimedia is a catalyst for social evolution. It js, "the 
architecture of the future" — or the language of the new era. » 



ferspace under construction 



imications networks and digital memorias will soon incorporate nearly all of 
representation and messages in circulation. At this point I would like to fcider 



the significance of the risks involved in developing these networks. Pol-and 
aesthetics confront one another within the unbounded construction of 
cyberspace. The perspective of collective intelligence is only one possible 
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approach, however. Cyberspace might also presage, or even incarnate, the 
terri tying, often inhuman future revealed to us by science fiction: the cataloging 
of th individual, the processing of delocalized data, the anonymous exercise of 
powei implacable techno-financial empires, social implosion, the annihilation of 
mera ory, real-time warfare among maddened and out-of-control clones. 
Nevertht less, a virtual world of collective intelligence could just as easily be 
as replel with culture, beauty, intellect, and knowledge as a Greek temple, a 
Gothic cath< dral, a Florentine palazzo, the Encydopedie of Diderot and 
d'Alembert, or th Constitution of the United States. A site that harbors 
unimagined language galaj fes, enables unknown social temporalities to 
blossom, reinvents the social bom perfects democracy, and forges unknown 
paths of knowledge among men. But t do so we must fully inhabil this site; it 
must be designated, recognized as a | tendal for beauty, thought, and new 
forms of social regulation. I would like to en this first part of the book with a 
discussion of the aesthetic dimension associate with engineering the social 
bond, which primarily focuses on the design of q berspace and the use of 
creative play within this new environment of commun cation and thought. 

Cyberspace. Of American origin, the word was used for the first time in 198 
by the science-fiction writer William Gibson in his novel NeuromancerJ Cybe 
space designates the universe of digital networks as a world of interaction and 
ai venture, the site of global conflicts, a new economic and cultural frontier. 
The currently exists in the world a wide array of literary, musical, artistic, even 
poli ical cultures, all claiming the tille of "cyberculture." Bui cyberspace refers 
less i the new media of information Iransmission ihan lo original modes of 
crealion ai navigalion within knowledge, and the social relations they bring 
aboul. The would include, in no particular order: hyptertext, ihe World Wide 
Web, inte active multimedia, video games, simulations, virlual realily, 
lelepresence, a.\i menled realily (whereby our physical environment is 
enhanced with networks sensors and intelligent modules), groupware (for 
collaborative activities), neur mimelic programs, artificial life, experl syslems, 
ele. All of these lools are coi bined in exploiling ihe molecular characler of 
digilized informalion. Vanen hybrids of ihe above lechnologies and 
convenlional media (telephone, film, lei vision, books, newspapers, museums) 
will come inlo existence in the near futui Cyberspace constitutes a vasl, 
unlimiled field, slill parlially indelerminale, whii shouldn't be reduced to only one 
of its many componenls. It is designed to i terconnect and provide an 



interface for the various methods of creation, recor ing, communication, and 
simulalion. 

While ihe Irue "greal works" remain lo be accomplished wilhin ihe univei 
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af digital information and at the new sites for the emergence of collective 
intelligence, we continue to encumber the landscape with cement, glass, and 
steel. We Nave built pyramids when we are in the process of again 
becoming nomads, when an architecture for a new exodus is needed. In the 
silence of thought, we will travel the digital avenues of cyberspace, inhabil 
weightless mansions that will now constitute our subjectivity. Cyberspace: 
urban nomad, software engi-neering, the liquid architecture of the knowledge 
space. It brings with it methods of collective perception, feeling, 
remembrance, working, playing, and being. It is in interior architecture, an 
unfinished system of intelligence hardware, a gyrat-ing city with its rooftops 
of signs. The development of cyberspace, the quintes-sential medium of 
communication and thought, is one of the principal aesthetic ind political 
challenges of the coming century. 
Digital interactive multimedia, for example, explicitly poses the question of 
iie end of logocentrism, the destitution of the supremacy of discourse over 

other 

nodes of communication. It is likely that human language appeared 

simultane- 

)usly in several forms: oral, gestural, musical, iconic, plastic, each individual 

ex- 

jression activating a given region of a semiotic continuum, bouncing back 

and 

brth from one language to another, from one meaning to another, following 

the 

•hizomes of signification, increasing the powers of mind as it traversed the 

body 

md its affects. The systems of domination founded on writing have isolated 

lan- 

;uage, established its mastery over a semiotic territory that has been cut 

up, 

>arceled out, and judged in terms of a sovereign logas. The appearance of 

hy- 

>ermedia, however, sketches an interesting possibility (among others that 

are 

ess interesting): A resurgence that lies well within the path opened by 

writing 



ills short of a triumphant logocentrism and moves toward the rediscovery of 

a 

leterritorialized semiotic plane. But such a resurgence will be enriched with 

the 

iowers of the text; it will be based on instruments that were unknown during 

the 

'aleolithic and are capable of bringing signs to life. Rather than limiting 

ourselves 

d the facile opposition between reasonable text and fascinating image, 

shouldn't 

r& attempt to explore the richer, subtler, more refined possibilities of thought 

and 

xpression created by virtual worlds, multimodal simulations, dynamic writing 

icdia? 

In evaluating these new technologies, should we limit ourselves to the con- 
spts of the information highway, telecommuting, interactive CDs, and virtual re- 
ity gaming that are presented for public consumption by the media? In 
doing ) we lose sight of the continuity between these spectacular phenomena 
and the ivisible, day-to-day use of existing intellectual technologies. When 
such new ichnologies are presented as unrelated phenomena, as objects 
fallen from the 



374 Fierre Levy 



sky, we lose sight of the open and dynamic system they form, their 
interconnec-tion in cyberspace, their contentious insertion in ongoing cultural 
processes. A e remain blind to the different possibilities they offer to human 
becoming, possibilities whose full scope is rarely perceived and which should 
be the subject o: deliberation, choice, and judgments of taste, rather than the 
fiefdom of technica-specialists. Even in terms of the apparatus of 
communication and thought, we are neglecting the dimension of interiority and 
collective subjectivity, ethics, and sensibility that even the most seemingly 
technical decisions imply. 



from design to implementation 

With respect to its relationship to future projects, cyberspace will assume th; 
form of a cultural attractor, which we can summarize as follows. 

3. Called, controlled, dismissed, distanced, combined, etc., no matterthey 
are orchestrated, messages, regardless of type, will now revolve arouncthe 
individual receiver (the opposite of the situation represented by themass 
media). 

4. The distinctions between authors and readers, producers and 

spectator?-creators and interpreters, will blend to form a reading-writing 
continuur.which will extend from machine and network designers to the 
ultimate re-cipient, each helping to sustain the activity of the others 
(dissoludon athe signature). 

3 . The distinction between the message and the work of art, envisaged as a 
nt-croterritory attributed to an author, is fading. Representation is now 
sur-ject to sampling, mixing, and reuse. Depending on the emerging 
pragman : -of creation and communication, a nomadic distribution of 
information w_ fluctuate around an immense deterritorialized semiotic 
plane. It is thereto r-natural that creative effort be shifted from the message 
itself to the mear A -processes, languages, dynamic architectures, and 
environments used for .1 implementation. 



Some of the questions that artists have been asking since the end of the 
ni:.: teenth century will thus become more urgent with the emergence of 
cyberspact-These questions are directly concerned with the question of the 
frame: the fm-its of a work, its exhibition, reception, reproduction, distribution, 
interpretatk». and the various forms of separation they imply. Under the 
present circumstanctt. however, no form of closure will be able to contain 
deterritorialization in o- 
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remis — a leap into a new space will be required. Mutation will occur in a so- 
:iotechnical environment in which works of art proliferate and are distributed, 
fet, is it reasonable to even speak of a work of art in the context of cyberspace? 
For the past several centuries in the West, artistic phenomena have been pre- 
ented roughly as follows: a person (artist) signs an object or individual 
message 

the work), which other persons (recipients, the public, critics) perceive, appre- 
iate, read, interpret, evaluate. Regardless of the function of the work (religious, 
ecorative, subversive, etc.) or its capacity to transcend function in search of the 
ore of enigma and emotion that inhabits us, it is inscribed within a conventional 
attern of communication. Transmitter and receiver are clearly differentiated 
id their roles uniquely assigned. The emerging technocultural environment, 
Dwever, will encourage the development of new kinds of art, ignoring the sep- 
ation between transmission and reception, composition and interpretation, 
evertheless, the ongoing mutation creales a realm of the possible that may 
never 

realized or only incompletely. Our primary goal should be to prevent closure 
>m occurring too quickly, before the possible has an opportunity to deploy the 
riety of its richness. With the disappearance of a traditional public, this new 
m of art will experiment with different modalities of communication and cre- 

Rather than distribute a message to recipients who are outside the process of 
ation and invited to give meaning to a work of art belatedly, the artist now at- 
ipts to construct an environment, a system of communication and production, 
jllective event that implies its recipients, transforms interpreters into actors, 
bles interpretation to enter the loop with collective action. Clearly the "open 
•k" prefigures such an arrangement. But it remains trapped in the hermeneu- 
jaradigm. The recipients of the open work are invited to fill in the blanks, 
ose among possible meanings, confront the divergences among their iter- 
ations. In all cases it involves the magnification and exploration of the pos- 
ities of an unfinished monument, a succession of initials in a guest book 
ed by the artist. But the art of implication doesn't constitute a work of art at 
ven one that is open or indefinite. It brings forth a process, attempts to open 
eer to autonomous lives, provides an introduction to the growth and habi- 



n of a world. It places us within a creative cycle, a living environment of 
h we are always already the coauthors. Work in progress? The accent has 
shifted from work to progress. Its embodiment is manifested in moments, 
s, collective dynamics, but no longer in individuals. It is an art without a sig- 
e. 

he classic work of art is a gamble. The more it transmutes the language on 
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which it rides, be it musical, plastic, verbal, or other, the more its author runj 
risk of incomprehension and obscurity. But the larger the stake — the degr< 
change or fusion to which its language is subject — the greater the potential j 
the creation of an event in the history of a culture. Yet this game of language, 
wager on incomprehension and recognition, is not restricted to artists al 
Each of us in our own way, as soon as we express ourselves, produces, re 
duces, and alters language. From singular utterances to creative listening, 
guages emerge and drift along the stream of communication, borne by thous; 
of voices that cali and respond to one another, take risks, provoke and deo 
hurling words, expressions, and new accents across the abyss of non-sensi 
this way an arust can appropriate an expression inherited from earlier general 
and help it evolve. This is one of the primary social functions of art: participz in 
the continuous invention of the languages and signs of a community. Bu 
creator of language is always a community. 

Radicalizing the classical function of the work of art, the art of implication ates 
tension and provides us with sign machines that will enable us to inveni 
languages. Critics may claim that we have been producing languages fon 
True, but without our awareness. To avoid trembling in the face of our owi 
dacity, to mask the void beneath our feet, or simply because this activity has 1 so 
slow that it has become invisible, or because it has had to encompass m* of 
people in constant motion, we have preferred the illusion of foundation, the 
price of this illusion has been our sense of defeat. Powerless before the 
guage of the absolute, overeome by the transcendence of the lagos, exhaust! 
the presence of the artist's inspired effusions, castigated and corrected, bea 
the weight of forgotten tongues, we falter beneath the exteriority of langt The 
art of implication, which can only give some idea of its true scope in cy space, 
by organizing cyberspace, is an art of therapy. It encourages us to ex] ment 
with the collective invention of a language that recognizes itself as such, in so 
doing, it points to the very essence of artistic creation. 

Having stepped out of the bath of life, far removed from their areas of c 
petence, isolated from one another, individuals finally "have nothing to say." 
difficulty lies in trying to comprehend them — in both the emotional and ti 
logical sense — as a group, in engaging them in an adventure in which they e 
imagining, exploring, and constructing sentient environments together. Ev live 



and real-time technologies play a role in this undertaking, the time ex] enced 
by the imagining community overflows the staccato, accelerated, qi punctual 
temporality of "interactivity." The inadequacy of the immediat amnesiac 
channel hopping, no longer leads to lengthy sequences of interp 
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tion, the infinite patience of tradition, which encompasses in a single sweep 
the ages of the living and the dead, and employs the quick currents of the 
present to erect a wall against time. In much the same way as madrepores 
erect coral reefs, commentary, strata upon strata, is always transformed into 
a subject for com-mentary. The rhythm of the imagining community resembles 
a very slow dance, a slow-motion choreography, in which gestures are slowly 
adjusted and respond with infinite precaution, in which the dancers gradually 
discover the secret tempi that will enable them to shift in and out of phase. Each 
learns from the others how to make their entrance in stately, slow, and 
complicated synchrony. Time in the intelligent community spreads itself out, 
blends with itself, and calmly gathers it-self together like the constantly 
renewed outline of the delta of a great river. The imagining collective comes 
into being so that it may take the time to invent the ceremony by which it is 
introduced, which is at the same time a celebration of ori-gin and origin itself, 
still undetermined. 

Employing all the resources of cyberspace, the art of implication reveais the 
priority of music. But how can a symphony be created from the buzz of 
voices? Lacking a score, how can we progress from the murmur of the crowd 
to a cho-rus? The collective intellect continuously questions the social contract, 
maintains the group in a nascent state. Paradoxically, this requires time: the 
time to make sure people are involved, time to forge bonds, to bring objects 
into being, shared landscapes ... the time to return. From the point of view of 
a watch or a calendar, the temporality of the imagining collective might seem 
displaced in time, in-terrupted, fragmented. But everything occurs within the 
obscure, invisible folds of the collective itself: the melodic line, the emotional 
tonality, the hidden inter-vals, the correspondences, the continuity that it 
weaves within the hearts of the individuals who compose it. 



for an architecture df deterritorialization 



Fhe artists who explore such alternatives may be the pathfmders of the new 
architecture of cyberspace, which will undoubtedly become one of the major 
arts )f the twenty-first century. The new architects could just as easily be 
engineers, ictwork or interface designers, software programmers, 
international standards irganizations, information lawyers, etc., as individuals 
with a background in tra-litional forms of art. In this field the most obviously 
"technical" choices will 
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tiave considerable political, economic, and cultural impact. We know that tradi- 
donal architects and urban planners have helped produce our material, 
practical, and even symbolic environment. In the same way, the sponsors, 
designers, and engineers of cyberspace will help produce the environraents 
of thought (sign systems, intellectual technologies), action (telecommuting, 
remote operation), and communication (access rights, rate policies) that will, to 
a large extent, struc-ture social and cultural developments. 

To guide the construction of cyberspace, to help us choose among the dif- 
ferent possible orientations or even imagine new ones, some criteria of ethical 
and political selection are needed, an organizing vision. Means that contribute 
to the production of a collective intelligence or imagination should be 
encouraged. In keeping with this general principle, I would suggest that we 
concentrate on the following: 

45. Instruments that promote the development of the social bond 
throughapprenticeship and the exchange of knowledge 

46. Methods of communication that are predisposed to acknowledge, inte 
grate, and restore diversity rather than simply reproduce traditional media- 
driven forms of distribution 

47. Systems that promote the emergence of autonomous beings, 
regardless ofthe nature of the system (pedagogical, artistic, etc.) or the 
beings involved(individuais, groups, works of art, artificial creatures) 

48. Semiotic engineering that will enable us to exploit and enhance, for 
thebenefit of the greatest number, the veins of data, the capital of skills, 
andsymbolic power accumulated by humanity 

In terms ofthe creation and management of signs, the transmission of 
knowledge, the development of living and thinking spaces, the best 
propaedeutic is ob-viously supplied by literature, art, philosophy, and high 
culture in general. Barbarism is born of separation. Contrary to what they 
may think, technicians I have a great deal to learn from humanists in this area. 
Likewise, those in the hu-1 manities must make an effort to employ the new 
tools, since they redefine work of intelligence and sensation. Lacking such 
interaction, we will ultimat produce nothing more than a meaningless 
technology and a dead culture. 



I am making a case here for an architecture without foundations, similar to: 
boat, with its system of practical oceanography and navigation. Not some be 
symbolic structure, analogous to a static image of the body or mind, the refle 
tion of a stable world. On the contrary, the architecture of the exodus will give n~ 
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> nomadic cosmos that travels the universe of expanding signs; it will bring >ut 
endless metamorphoses of bodies; within the fissure of flesh and time, it 
[ dispatch its fleets toward the inviolate archipelagos of memory. Far from en-dering 
a theater of representation, the architecture of the future will assemble s of icons 
to help us cross the seas of chaos. Attentive to the voice of the col-ive brain, 
translating the thought of plurality, it will erect sonorous palaces, ;s of voices and 
chants, instantaneous, luminous, and dancing like flames. 

— translated by robert bononno 
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Myst. Courtesy ofCyan Worlds, Inc. 

"In games, therefore, we have a chance to enact our most basic 
relationship to the world — our desire to prevail over adversity, to 
survive our Inevitable defeats, to shape our environment, to 
master complexity, and to make our Uves fit together like the 
pieces of a jigsaw puzzle. " 
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Janet H. Murray isoneof the leading scholars in interactive game design. Au-' of 
the influential book, Hamlet on the Holodeck: The Future of Narrative in 
Cyberspace, work has been widely embraced as a roadmap by artists and 
designers shap-the future of digital narrative. In her writing Murray identifies the 
formal par-jms used by leading game developers; from these observations she 
synthesizes iherent approach to computer-based storytelling. Her critical 
analyses of in-ictive narrative, role-playing games, interactive television, and 
large-scale timedia information spaces have shed light on design strategies 
and dra-,ic methodologies for new media artists, designers, and educators. Of 
central importance to Murray is the concept of "agency," which she debes as 
"the satisfying power to take meaningful action and see the results of decisions 
and choices." In this chapter from her book she discusses the game nents that 
contribute to this empowerment of the participant — spatial navi-on, labyrinthine 
puzzles, contests, and extensibility. She also reviews such key gational structures 
as the "soivable maze/'the "tangled rhizome/'and the irney story." Murray 
emphasizes that, regardless of form, for an interactive ative to besuccessful it 
mustengagethefull participaron of the audience — 3 use her preferred term, the 
"interactor." 

Vlurray notes that interactive multimedia has an architectonic aspect that htens 
agency by inviting the interactor to navigate jts space. But while the icipant can 
make important choices about the direction of an interactive nar-'e, the artist 
remains the "procedural author," creating "not just a series of es but a world of 
narrative possibilities/The exploration of a world, and the unter between the 
consciousness of the interactor and the artist who designs world, is what leads 
to, in Murray's words, "the pleasures of agency." » 



6 more realizad the immersive environment, the more active we want to be n it. 
When the things we do bring tangible results, we experience a delight ictronic 
environments — the sense of agency. Agency is the satisfying power :e 
meaningful action and see the results of our decisions and choices. We ex-to feel 
agency on the computer when we double-click on a file and see it 
before us or when we enter numbers in a spreadsheet and see the totals ust. 
However, we do not usually expect to experience agency within a nar-; 
environment, ven in the rare circumstances when we are invited to participate 
in a tradi- 

narrative form, our participation is circumscribed in a way that generally 



382 JanetMurray 



limits our sense of agency. For instance, if the audience at a performance of P<Pan 
decided that Tinkerbell is a pest and refused to clap her back to life, the jwould 
come to a halt. The participatory dinner theater plays that simulate an 1ian 
wedding or an Irish wake or a Jewish funeral encourage audience partidtion by 
keeping the plot to a minimal level and the dialogue with the audiencsocial 
formulas appropriate to distant friends of the family. When the groorsuch a play 
leans down to kiss me as a guest at his wedding, I can congratuhim and warn 
him about staying away from his ex-girlfriend now that he is nried, but I cannot 
really influence his behavior. When audience members arecluded in the story, 
they serve only as the butt of ajoke. They may be accuseadultery by the priest or 
shot by a mahoso relative. The slender story is desigto unfold in the same way no 
matter what individual audience members majto join the fun 

Because of the vague and pervasive use of the term interactivity, the pleaj of 
agency in electronic environments is often confused with the mere abilit move a 
joystick or click on a mouse. But activity alone is not agency. Foi stance, in a 
tabletop game of chance, players may be kept very busy spim dials, moving 
game pieces, and exchanging money, but they may not have true agency. The 
players' actions have effect, but the actions are not chosen the effects are not 
related to the players' intentions. Although gamemakers so times mistakenly 
focus on the number of interactions per minute, this numb a poor indicator of the 
pleasure of agency afforded by a game. Some games, chess, can have relatively 
few or infrequent actions but a high degree of age since the actions are highly 
autonomous, selected from a large range of poss choices, and wholly determine 
the course of the game. 

Agency, then, goes beyond both participation and activity. As an aestl 
pleasure, as an experience to be savored for its own sake, it is offered to a lira 
degree in traditional art forms but is more commonly available in the structi 
activities we cali games. Therefore, when we move narrative to the compute] 
move it to a realm already shaped by the structures of games. Can we imagi 
compelling narrative literature that builds on these game structures without b 
diminished by them? Or are we merely talking about an expensive way to rev 
Hamkt for the pinball machine? 



the pleasures of navigation 

One form of agency not dependent on game structure yet characteristic of i tal 
environments is spatial navigation. The ability to move through virtual I 
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ipes can be pleasurable in itself, independen! of the conteni of the spaces. A ;nd 
of mine whose son is an avid Ninlendo player complains that when he tries t the 
games he is annoyed at having to be fighting all the time, since the com-t is an 
unwelcome distraction from the pleasure of moving around the unfold-; spaces of 
the ma/e. For my friend, videogames are about exploring an initely 
expandable space. Similarly, new explorers of the World Wide Web d 
themselves entranced with the ability to leap around the world, following ks from 
one home page or Web site to the next mostly for the pleasure of the >eated 
arrivals. The navigational pleasures are richly exploited by the many ms of 
labyrinths, from Zor/fc-like dungeons to informational webs, that fill cy-»space. All of 
them allow us to experience pleasures specific to intentional dgation: orienting 
ourselves by landmarks, mapping a space mentally to match • experience, and 
admiring the juxtapositions and changes in perspective that ive from moving 
through an intricate environment. 

This participatory pleasure is not unlike the enjoyment people find in the or-dzed 
sport of "orienteering," where players follow a series of geographical es across 
a large and complex terrain, such as a portion of the Maine woods lowntown 
Boston. Construing space and moving through it in an exploratory ' (when done 
for its own sake and not in order to find the dentist's office or right airport gate) is 
a satisfying activity regardless of whether the space is real irtual. Electronic 
environments offer the pleasure of orienteering in two very srent configurations, 
each of which carries its own narrative power: the solv-; maze and the tangled 
rhizome. 



ie story in the maze 

t-like puzzle dungeons and maze-based combat videogames derive from a >ic 
narrative of adventure whose roots are in antiquity. It was Daedalus who t King 
Minos of Crete a labyrinth to contain the deadly Minotaur. The hor-: beast 
required the yearly sacrifice of Athenian youths and maidens, whom voured, 
until the hero Theseus arrived to slay it. In the story, Ariadne, the fhter of the 
king, fell in love with Theseus and gave him a sword to kill the t and a thread to 
find his way out again. Minos's maze was therefore a fright-g place, full of 
danger and bafflement, but successful navigation of it led to : rewards. 
The adventure maze embodies a classic fairy-tale narrative of danger and sal-n. 
Its lasting appeal as both a story and a game pattern derives from the ling of 
a cognitive problem (finding the path) with an emolionally symbolic 
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pattern (facing what is frightening and unknown). The maze story celebrales 
combination of intelligence and courage, and it depicts romantic love as the < 
ment that provides the hope that brings the hero into the confrontation and bj 
out again to safety. Like all fairy tales, the maze adventure is a story about survi 
The maze is a road map for telling this story. 

As a format for electronic narrative, the maze is a more active version of i 
immersive visit. Maze-based stories take away the moving platform and n 
the passively observant visitor into a protagonist who must find his or her o 
way through the fun house. A typical maze-based puzzle game sends you. I 
player, through a multitiered space vaguely resembling an Arabian Nig 
palace. You operate an avalar who walks through the palace rooms, whose ti 
floors and ornately decorated corners often hide treasures that are tricky to p 
ceive. The palace is full of informants, who speak in text bubbles and whom y 
reply to from menus, and you must negotiate with them carefully, offering thi 
icons representing money or other valuables. A mysterious peddler on one 
the lower leveis holds a talisman needed to get into the highest chamber. "V 
must have it with you while you stand on a special spot that is hidden in the p 
terning of the floor. If you forget to get it, you must retrace your steps throu 
many perils. The game is like a treasure hunt in which a chain of discoveries a 
as a kind of Ariadne's thread to lead you through the maze to the treasure at I 
centen 

This kind of narrative structure need not be limited to such simplistic c< 
tent or to an explicitly mazelike interface. In the right hands a maze story coi 
be a melodramatic adventure with complex social subtexts. For instance, insti 
of a fairy-tale palace it could be set in a Kafkaesque city where the secret pol 
are rounding up and deporting citizens with the wrong kind of papers. The p 
tagonist's role would be to save them, a task that would require navigad 
through the corridors of power and through underground hiding places, elal 
rately conducted negotiations, clever manipulation of bureaucrats, and sp 
second timing. The characters in the menacing world could be subtly portrayt 
in either graphics with text bubbles or in video segments. Saving people mij 
involve horrifying choices, perhaps implicating the protagonist in the corrupri of 
the violent world. The maze could be composed not only of spatial twists I of 
moral and psychological choices.Just as it hard to see where a tangle of viro 
corridors is leading, so too would it be hard to foresee the consequences of yo 
actions and to determine what to value and whom to trust. Just as Kafka used I 
conventions of the fable to convey the profound depersonalization of modern 1 
and Art Spiegelman used the format of the comic book to tell the story of his 
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.er's Holocaust experiences, a digital artist might use the structure of the 
acknture maze to embody a moral Individual's confrontation with state- 
sanctioned olence. 

Whether an adventure maze is simple or complex, it is particularly suited to 
e digital environment because the story is tied to the navigation of space. As I 
ove forward, I feel a sense of powerfulness, of significan! action, that is tied to 
y pleasure in the unfolding story. In an adventure game this pleasure also 
feels :e winning. But in a narrative experience not structured as a win-lose 
contest e movement forward has the feeling of enacting a meaningful 
experience both nsciously chosen and surprising. However, there is a drawback 
to the maze ori-tation: it moves the interactor toward a single solution, toward 
finding the one ly out. The desire for agency in digital environments makes us 
impatient when r options are so limited. We want an open road with wide 
latitude to explore d more than one way to get somewhere. We want the 
"pullulating" web that 

rges described, constantly bifurcating, with every branch deeply explorable. 



-\PTURE OF THE UHIZOME 

e second kind of digital labyrinth, which has arisen from the academic liter- 
community, is the postmodern hypertext narrative. Full of wordplay and in- 
srminate events, these labyrinths derive not from Greek rationalism but from 
tstructuralist literary theory and are unheroic and solutionless. Like a set 
of ;x cards that have been scattered on the floor and then connected with 
mul-2 segments of tangled twine, they offer no end point and no way out. 
Their hetic vision is often identified with philosopher Gilies Deleuze's 
"rhizome," ber root system in which any point may be connected to any 
other point. 1 fuze used the rhizome root system as a model of connectivity 
in systems of s; critics have applied this notion to allusive text systems that 
are not linear a book but boundaryless and without closure. Stuart 
Moulthrop, a theorist electronic fiction writer, states it this way: 

leen from the viewpoint of textual theory, hypertext systems appear as the 
prac-ical implementation of a conceptual movement that... rejects 
authoritarian, "lo-ocentric" [i.e., truth-affirming] hierarchies of language, 
whose modes of peration are linear and deductive, and seeks instead 
systems ofdiscourse that dmit a plurality of meanings where the operative 
modes are hypothesis and in-irpretive play. 2 
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The postmodern hypertext tradition celebrales the indeterminate text as a li 
eration from the tyranny of the author and an affirmation of the reader's freedc of 
interpretation. But the navigational software designed specifically for this pu pose 
and celebrated by many proponents of literary hypertext is anything but en 
powering to the reader, even in comparison to the earliest Web browsers. 3 F< 
instance, it offers the navigating reader no way to mark links as having been ; 
ready taken, and no way to mark a lexia so it can be easilyjumped back to. Mar of 
the stories written in this framework do not even mark which words are ha links 
within the lexia text. Instead, the reader has to click on a pop-up display cryptic 
link names. Moulthrop's own Victory Carden, which is perhaps the mot coherently 
structured literary hypertext, contains a clever overview map of th major story 
clusters, which are arranged like a Borgesian garden labyrinth. B readers cannot 
easily return to the overview in order to get a sense of where th are or how much 
is left to read. In trying to create texts that do not "privilege any one order of 
reading or interpretive framework, the postmodernists are pn ileging confusion 
itself. The indeterminate structure of these hypertexts fru trates our desire for 
narrational agency, for using the act of navigation to unfo a story that flows from 
our own meaningful choices. 

But the unsolvable maze does hold promise as an expressive structure. WalkJ 
ing through a rhizome one enacts a story of wandering, of being enticed in con- 
flicting directions, of remaining always open to surprise, of feeling helpless td 
orient oneself or to find an exit, but the story is also oddly reassuring. In the rhi- 
zome, one is constantly threatened but also continuously enclosed. The fact that 
the plot will not resolve means that no irreparable loss will be suffered. The nar- 
rator of Michaeljoyce's Afternoon will not have to confront the fact of the morn-i 
ing's fatal accident so long as the afternoon's evasive wanderings continue, andj 
the reader of Victory Garden does not have to accept the death of an appealing 
character so long as there are multiple paths to explore, including some that lead 
to alternate realities in which she comes back home from the war. In both stories 
the reader is protected from feeling the irreversibility of death by the fact that the 
stories do not have to end there. 

The boundlessness of the rhizome experience is crucial to its comforting 
side. In this it is as much of a game as the adventure maze. In fact, it reminds me 
of a particular game my son William invented at about age five. At his own ini- 
tiative he one day drew a large game board, assembled dice and playing pieces, 
and invited his father to join him in an inventively improvised game with ever- 
changing and ever more elaborate rules. After two hours of this surreal activity, 
my husband became restless and began asking every five minutes or so if the 
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ime was almost oven William responded by calmly walking into the kitchen, here 
I was sitting, and asking me to write his father the following note: 

DEAR DAD— THIS GAME WILL NEVER END. WILLIAM 

he rhizome has the same message. As we navigate its tangled, anxiety-laden 
iths, enclosed within its shape-fitting borders, we are both the exasperated par-it 
longing for closure and separation and the enthralled child, lingering forever an 
unfolding process that is deeply comforting because it can never end. 



riviNG shape to anxiety 

Dth the overdetermined form of the single-path maze adventure and the under- 
itermined form of rhizome fiction work against the interactor's pleasure in navi-.tion. 
The potential of the labyrinth as a participatory narrative form would em to lie 
somewhere between the two, in stories that are goal driven enough to dde 
navigation but open-ended enough to allow free exploration and that dis-ay a 
satisfying dramatic structure no matter how the interactor chooses to tra-rse the 
space. 

The key to creating an expressive fictional labyrinth is arousing and regulat-f the 
anxiety intrinsic to the form by harnessing it to the act of navigation. Sus-nse, fear 
of abandonment, fear of lurking attackers, and fear of loss of self in the 
.differentiated mass are part of the emotional landscape of the shimmering :b. 
Moving through the space can therefore feel like an enactment of courage d 
perseverance, like Gary Cooper's striding through the town in High Noon. 
>mputer gamers often experience shivers of physical fear as they approach an 
opened door in a text-based or graphics-based labyrinth. The drama of sus- 
nseiul approach does not have to be tied to combat or to jack-in-the-box ef-ts. It 
can also have the feeling of a determination to face the truth, to stare »ectly at 
the threatening beast. It can be experienced by the navigating .der/viewer as 
well as by the player/protagonist. 

One such use of the labyrinth as a means of evoking and controlling terror is :ory 
formal increasingly used by my fiction students, a story I have come to cali "violence 
hub." Writers will place an account of a violent incident, often a real invented 
newspaper article, at the center of a web of narratives that explore it m multiple 
points of view. A helicopter accident near MIT, a robbery in a ivenience store, a 
canoeing fatality — these are all examples of an act of sudden lence that have 
served as the center of such a story web. The incident ac- 
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count itself is usually bare but evocative. People have died in violent but su 
prising circumstances, we are told. The incident happened at a particular pla< 
and time involving a particular group of people. The names in the account or i 
a diagram of the site of the accident lead outward with hot links to the satellite fii 
that tell us how the incident appeared to the various people involved — the p 
petrator, the witnesses, the rescuers, the victims, the survivors. 

The proliferation of interconnected files is an attempt to answer the perei 
nial and ultimately unanswerable question of why this incident happened. For ii 
stance, one convenience store robbery labyrinth follows the robber, the clerk, ti 
owner, and the cop (who shoots the robber) back through the events leading 
u to the shootout and forward again into the moment of violence. 4 Reading it m 
feel sympathy with all of them, and we can see how they appeared to one 
anothe A web story of a teenager who drowned on a white-water canoeing trip 
descrita the traumatic experience from the points of view of the friends he was 
traveliu with, the operators of the canoeing company, the emergency 
medies, and ti family members receiving the dreadful phone cali. These 
violence-hub stories d not have a single solution like the adventure maze or a 
refusal of resolution lil the postmodern stories; instead, they combine a clear 
sense of story structui with a multiplicity of meaningful plots. The navigation of 
the labyrinth is like pa< ing the floor; a physical manifestation of the effort to 
come to terms with ti trauma, it represents the mind's repeated efforts to keep 
returning to a shockiil event in an effort to absorb it and, finally, get past it. The 
retracing of the situs tion from different perspectives leads to a continual 
deepening in the reader's ui derstanding of what has happened, a deepening 
that can bring a sense < resolution but one that allows for the complexity of 
the situation and that leavi the moment of shock unchanged and still central. 
A linear story, no matter how complex, moves toward a single encompassin 
version of a complex human event. Even those multiform stories that offer mu 
tiple retellings of the same event often resolve into a single "true" version — th 
viewpoint of the uninvolved eyewitness or the actual reality the protagonisl 
wind up in after the alternate realities have collapsed. A linear story has to en 
in some one place: the last shot of a movie is never a split screen. But a mult 
threaded story can offer many voices at once without giving any one of them th 
last word. This is a reassuring formal for encountering a traumatic event 
becaua it allows plenty of room for conflicting emotions. It lets us disperse 
complex, ir tense reacu'ons into many derivative streams so that we do not 
have to feel the fu flood of sorrow all at once. The multithreaded web story 
achieves coherent dn matic form by shaping our terror into a pattern of 
exploration and discovery. 
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phejourney story and the pleasure >f problem solving 

'he navigational space of the computer also makes it particularly suitable for 
>urney stories, which are related to mazes but ofler additional opportunities for 
iercising agency. Journey stories date back to oral storytelling, from the fairy-tale 
onvention of setting out from home to find one's fortune to the voyages of 
Idysseus and Sinbad. It is a universal archetype recognizable across all the 
vari-ions of culture, author, and medium. After the invention of the printing 
press, le journey story was reinvented as the picaresque novel, exemplified 
by Don 'uixote, Moll Flanders, Tom Jones, Huckleberry Finn, and Catcher in 
the Rye. fith the invention of the movie camera, the journey story was again 
revived, and s variations include everything from the comic (e.g., Buster 
Keaton's The Gen-*at) to the cowboy melodrama (e.g., Stagecoach, The 
Searchers) and the feminist iddy film (Thelma andLouise). When television 
carne along, journey stories Vagon Train, Route 66, The Fugitive, and, of 
course, Star Trek) were among the ost successful series. 
Moving thejourney story from the fairy tale to the novel meant moving it from 
iymbolic realm of universal actors (a king, a wicked stepmother) to a particu- 
ized social world and a particular time and place. In the novel the cruel things 
it happen to the hero are often treated as instances of a specific social 
injustice, 

e the English Poor Laws, rather than as the work of a generic antagonist like a 
5 bad wolf. Moving thejourney to the movies opened up the visual dimension 
the archetype. Journey films often emphasize exotic landscapes, foreign cul- 
•es, and the lure of open spaces. Since television is best at portraying interior 
imas and family-size social units, 5 journey stories on TV generally focus on a 
:cession of small commumties or even replace the hero and sidekick with an 
ire traveling community, as on Star Trek. On the computer thejourney story 
phasizes navigation — the transitions between difieren! places, the arrivals and 
Dartures — and the how-to's of the hero's repeated escapes from danger. 
One of the consistent pleasures of the journey story in every time and every 
dium is the unfolding of solutions to seemingly impossible situations. We 
.ch each new situation along the road and wonder how the hero will escape a 
.ring or a hanging or a forced marriage or jailing. When Odysseus foolishly al- 
s himself to be captured by the Cyclops, a huge, one-eyed man-eater, he is pre- 
ted with a life-and-death riddle. The situation is carefully described so that 
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it seems that he has no chance of getting out. The Cyclops is a brutal and 
heart less creature who brags that he is unafraid of Zeus and therefore free to 
do as ht picases. Every night two more of Odysseus's men are eaten; the 
survivors knoM they must get out soon or die. They could kill the monster as 
he sleeps, but tru cave is barred with a stone too heavy for them to move; if 
they kill him, they wJ never get out again. The Cyclops's routine is unvaried: he 
goes ofFwith the sheej in the morning, closing the cave behind him, and 
comes back at night with thi sheep, ready for a dinner of Greek sailors. Then 
Odysseus (who is narrating th( story) tells us how he solves the problem. He 
prepares some wine. He prepara a battering ram and gathers a group of 
strong helpers. He tells the giant his nam< is "Nobody" and gets him very 
drunk. When the Cyclops falls asleep, Odysseui and his men heat the 
battering ram in the fire and thrust it into his one terribl< eye. Now the giant is 
blinded, but how will the Greeks get out? While the Cyclop is raging, Odysseus 
separales the sheep into groups of three and places each a his men under 
the middle sheep and himself beneath the strongest ram in th< flock. Finally, 
the Cyclops lets out his sheep, as Odysseus has seen him do everj morning, 
and out go all the Greeks with them. And when the wounded Cyclopi cries out 
for help from his neighbors, whom does he say is tormenting himi "Nobody." 
Odysseus's description is constructed so that we can enjoy each in dividual 
step and gain increasing pleasure as the overall plan becomes clear. Th< story 
is as much a riddle as Oedipus's, but the answer to the riddle is not in a sin gle 
word; it is in a series of beautifully orchestrated steps, an elegant algorithm foi 
defeating giants. 

Computer-based journey stories offer a new way of savoring exactly this plea 
sure, a pleasure that is intensified by uniting the problem solving with the activi 
process of navigation. On the computer the dramatic situation of capture and 
es cape can be simulated by keeping the player within a confined space until 
the so lution to a puzzle is found. These puzzles are most satisfying when the 
action have a dramatic appropriateness, when they serve as a way of 
increasing our be lief in the solidity and consistency of the illusory world. For 
instance, in Myst thi wizard's island includes an elevator hidden in a giant 
tree and operated by j nearby control panel. In addition to solving the puzzle 
of the panel, the interac tor must move efficiently through the space to get to 
the elevator at just the righ moment in its descent. The concreteness with 
which the space is detailed make the sequence feel not like a test of 
coordination but like a dramatic moment. B; contrast, in the computer game 
The Seventh Guest, the player is asked to cut uj a cake into enough 
segmenta to match the number of murder victima. The puz zle is a satisfying 
one, but since there is no one there to eat the cake, the actioi takes us outside 
the immersive world instead of reinforcing our belief in it. 
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The most dramatically satisfying puzzles are those that encourage the interior 
to apply real-world thinking to the virtual world. For instance, a computa-Dnally 
sophisticated MIT student who is also an expert gamer instanced a irticular 
dramatic moment from the text-based Zork //as among his lifetime fa->rites: The 
story involves a dragon that is slow to rouse but always lethal if you :rsist in 
fighting him. Elsewhere in the dungeon is a wall of ice that is impossi-e to pass. 
What you must do is attack the dragon enough to get his attention — it not so 
much that he "toasts" you — and then run and head for the wall of ice. he dragon 
follows, sees his reflection in the ice, and thinks it is another dragon, e rears up 
and breathes fire at his enemy; as he does so, the ice melts, drown-g the dragon 
and eliminating the obstructing wall. 6 Like Odysseus in the Cy->ps's cave, the 
player escapes by outsmarting a ferocious monster using only the iterials at hand. 



ames into stories 

mes seem on the face of it to be very different from stories and 
to offer oppos- 

; satisfactions. Stories do not require us to do anything except to 
pay attention 

hey are told. 7 Games always involve some kind of activity and are 
often focused 

the mastery of skills, whether the skill involves chess strategy or 
joystick twitch- 
Games generally use language only instrumentally 
("checkmate," "ball four") 

icr than to convey subtleties of description or to communicate 
complex emo- 

is. They ofFer a schematized and purposely reductive vision 
of the world. 

st of all, games are goal directed and structured around turn 
taking and keep- 
score. All of this would seem to have nothing to do with stories. 
In fact, narrative satisfaction can be directly opposed to game 
satisfaction, as 

endings of Myst, widely hailed as the most artistically successful 
story puzzle 

te early 1990s, make clear. The premise of the Myst story is the 
confinement 

vo brothers, Sirrus and Achenar, in magical books that 
serve as a dungeon. 

ough a video window we can see them in their 
imprisonment and hear them 



ng to us in short, staticky segments. Each one warns us 
about the wickedness 

e other and asks us to rescue him. The brothers can be 
freed only by heroic 

rs of problem solving by the player, who must journey to 
four magical lands 

fes and bring back a single page from each of them for 
either Sirrus or 

mar. Each time the player gives one of the brothers a 
magic page, he re- 
ds with a slightly clearer video segment. At the end of the 
game, when most 

e puzzles have been solved, the player has most likely 
gone to each land 

: in order to gather both sets of pages and to hear all of the messages from 
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both brothers. At this point we are faced with a dramatic choice. The last magi; 
page will release one or the other of them from the book. Which is it to be? 

The game is well designed in that all the evidence on which to base a 
decision is, as in any good detective story, available to the player. Exploring the 
var-ious lands reveais — through accusatory notes, hidden corpses, imperia- 
furnishings, desolated landscapes, and multiple instruments of torture and 
de-struction — the villainy of both brothers. The secret of the game is that 
although both brothers are evil, their father, Atrus, is alive and — with some 
more puzzle solving — can be found and rescued. The "winning" ending 
involves locating tht good wizard Atrus and remembering to bring with you the 
magical Item that wiE free him from captivity. This is a satisfyingly fair yet 
challenging mystery plot. 

Yet surprisingly, the "losing" endings of the game are much more satisfying 
than the winning ending. In the winning ending one finds a beautifully renderec 
but dramatically inert video cutout of Atrus superimposed on a backdrop of 
very shallow fantasyland. Unlike all the other lands visited during the game, thii 
one is not really explorable and offers no pleasures of manipulation. It is a 
deac end. The ending in which you get to the wizard but forget to bring him the 
mean* of escape is more dramatic, because he gets quite angry at you. But the 
most dramatically satisfying endings are the near-identical losing branches, 
which are the result of choosing to rescue either of the evil brothers. The 
moment you releas* either Achenar or Sirrus from imprisonment, he will 
mockingly turn on you anc \ockyou in the very same dungeon from which he 
has escaped! The visual efieci is simple but brilliantly effective because it 
reverses your perspective. Througr-out the game you have peered into each 
brother's dungeon through a statk-ridden, credit-card-size window 
embedded within the parchment page of ar enchanted book. The brothers' 
immobility has been marked by the fact that yot could see little more than their 
faces. Now you are looking out through a similarH staticky window set into a 
totally black screen. Through the window you can set the evil brother now 
exultantly standmg and moving around while looking dowTi at youjust as you 
had looked down at him. A game that marked a breakthrougi: in ease of 
navigation appropriately ends by immobilizing the player. 
The superiority of the losing endings of Myst suggests a basic opposition br 
tween game form and narrative form. How can we tell significant stories in i 
form that always has to end happily? How can we impose endings that yic!: 
complex story satisfactions on a form that is based on win/lose simplicity? Mar. 
would argue that computer-based narrative will always be gamelike and that 
suci satisfactions are therefore inevitable. But when looked at more closely, 
games and stories are not necessarily opposed. 
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\mes as symbolic dramas 

ame is a kind of abstract storytelling that resembles the world of common ex- 
ience but compresses it in order to heighten interest. 8 Eveiy game, electronic 
itherwise, can be experienced as a symbolic drama. Whatever the conteni of 
game itself, whatever our role within it, we are always the protagonists of the 
ibolic action, whose plot runs like one of the following: 

3. I encounter a confusing world and figure it out. 

4 . I encounter a world in pieces and assemble it into a 
coherent whole. 

5 . I take a risk and am rewarded for my courage. 

6 . I encounter a difficult antagonist and triumph over 
him. 

7 . I encounter a challenging test of skill or strategy and 
succeed at it. 

8 . I start offwith very little of a valuable commodity and 
end up with a lot ofit (or I start offwith a great deal of a 
burdensome commodity and get ridof all of it). 

9 . I am challenged by a world of constant unpredictable 
emergencies, and Isurvive it. 

iven in games in which we are at the mercy of the dice, we are still enacting 
aningful drama. Playing purely luck-based games is captivating because we 
lodeling our basic helplessness in the universe, our dependence on unpre- 
ble factors, and also our sense of hopefulness. The people who line up at my 
iborhood convenience store for lottery tickets can be seen either as dupes 
risk takers engaging in a playful ritual of faith in the benevolence of forces nd 
their control. In fact, even when we lose, we are still part of the symbolic 
.a of the game. In that case the plots might go like this: 

I fail at an important test and suffer defeat. 

I decide to try again and again until I fmally succeed. 

I decide to win by cheating, that is, by acting outside the rules, because au- 
thority is meant to be flouted. 

I realize that the world is rigged against me and others like me. 

games, therefore, we have a chance to enact our most basic relationship to 

arid — our desire to prevail over adversity, to survive our inevitable defeats, 
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to shape our environment, to raaster complexity, and to make our lives fu : 
gether like the pieces of a jigsaw puzzle. Each move in a game is like a plot 
even in one of these simple but compelling stories. Like the religious 
ceremonies ;• passage by which we mark birth, coming of age, marriage, and 
death, games an ritual actions allowing us to symbolically enact the patterns 
that give meaning u our lives. 

Games can also be read as texts that offer interpretations of experience. I 
instance, the board game Monopoly can be read as an interpretation of 
capiui-ism, an enactment of the allures and disappointments of a zero-sum 
economv m which one gets rich by impoverishing one's neighbors. Or it can be 
read as a p;:-terned expression of our knowledge that success in life is always 
the result of botn planning and chance. When we play Monopoly, we are taking 
part in a structured drama that offers, in addition to its win/lose ending, 
moments in which we giv A expression to our ambition, greed, and 
benevolence and our tendencies to risks and exploit others. Even a game 
with no verbal conteni, like Tetris wildly popular and powerfully absorbing 
computer game of the early 1990s, has clear dramatic conteni. In Tetris, 
irregularly shaped objecls keep falling from cha lop of the screen and 
accumulating at the bottom. The player's goal is to guice each individual 
piece as it falls and position it so that it will fit together with other pieces and 
form a uniform row. Every time a complete row forras, it disap-pears. Instead of 
keeping what you build, as you would in a conventional jigsaw puzzle, in Tetris 
everything you bring to a shapely conclusion is swept away frora you. This 
game is a perfect enactment of the overtasked lives of Americana in the 1990s 
— of the constant bombardment of tasks that demand our attention and tha we 
must somehow fit into our overcrowded schedules and clear offour desks ii 
order to make room for the next onslaught. 9 

If the same spatial ideas behind the movement of the colored shapes iu 
Tetris — relentless activity, misfits and tight couplings, order and chaos, crowdin| 
and clearing — are represented in a dance, we automatically associate them 
wid| ordinary human experience, because we see human beings enacting 
them, h the computer game the interactor is the dancer and the game 
designer is th< choreographer. The screen objects are like a symbolic 
language for inducing ofa activity. So while we experience the game as being 
about skill acquisition, we an drawn to it by the implicit expressive conteni of 
the dance. Tetris allows us tt symbolically experience agency over our lives, It 
is a kind of rain dance for th» postmodern psyche, meant to allow us to enact 
control over things outside oui power. 

Games are recreational because they offer no immediate benefit to our sur 
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vival. Yet game-playing skills have always been adaptive behaviors. Games tradi- 
onally ofFer safe practice in areas that do have practical value; they are rehearsals 
ir life. Lion cubs roughhouse with one another in order to grow to be predators, 
small children still play hide-and-seek, a good way of training hunters, and ring- 
round-a-rosy, a good way of practicing cooperation and coordinated behaviors, 
older children in our society are understandably drawn to pitting themselves 
against machines. The violence and simplistic story structure of computer skill 
ames are therefore a good place to examine the possibilities for building upon le 
intrinsic symbolic conteni of gaming to make more expressive narrative forras. 



phe contest story 

The most common form of game — the agon, or contest between opponents — is 
also the earliest form of narrative. This is not surprising since opposition is one of 
the most pervasive organizing principles of human intelligence and language. 10 ast 
as we automatically organize the temporal and spatial world into opposing 
haracteristics (night/day, up/down, right/left), so too do we look at the things lat 
happen in the world in terms of struggles between opposites (God/Satan, 
lale/female, Cain/Abel, Jews/Gentiles). The Greek word agon refers to both 
:hletic contests and to dramatic conflicts, reflecting the common origin of games id 
theater. A simple shoot-'em-up videogame, then, belongs to the extremely road 
dramatic tradition that gives us both the boxing match and the Elizabethan tvenge 
play. 

Most of the stories currently told on the computer are based on the structure also 
contest of skill. The interactor is given the role of a fighter or detective of some 
sort and is pitted against an opponent in a win/lose situation. From their 
beginnings in the 1970s, computer games have also developed multiple repre- 
sentations of the opponent, who may be another human player (as in the first 
videogame, Pong), a character embedded in the story (as in Pacman), and the pro- 
grammer or game designer implicit in the game (as in ZorK). Contest games have 
also developed at least three different ways of situating the player: we can watch 
from a spectator perspective while operating our own avalar character or space- 
ship (as in Mortal Kombat); watch from a situated perspective while operating a 
character (as in Rebel Assault, where we see the vehicle we are operating as if 
are followingjust behind it with a movie camera); or, most immersively, watch and 
act from a situated first-person viewpoint, as in Doom, where we see the land- 
ape of the game and our opponents coming toward us as if we are really pre- 
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sent in space. These gaming conventions orient the interactor and make the 
tion coherent. They are equivalent to a novelist's care with point of view or a| 
rector's attention to staging. 

Fighting games have also developed a sure-hre way of combining agen 
with immersion. The most compelling aspect of the fighting game is the tight y 
ceral match between the game controller and the screen action. A palpable cli 
on the mouse or joystick results in an explosion. It requires very little imagil 
tive effort to enter such a world because the sense of agency is so direct. T 
imaginative engagement is even stronger with an arcade-style interface that / 
you sit in a brightly painted model of a spaceship or fire a toy gun. . . . 



the interactor as author 

One of the key questions that the practice of narrative agency evokes is, To wl 
degree are we authors of the work we are experiencing? Some have argued (w 
either elation or horror) that an interactor in a digital story — not just the imp 
vising MUDder, but even the navigating reader of a postmodern hypertext — is ( 
author of the story. This is a misleading assertion. There is a distinction betwe 
playing a creative role within an authored environment and having authorship 
the environment itself. Certainly interactors can create aspects of digital stor 
in all these formais, with ihe grealest degree of creative aulhorship being of 
ihose environments that reflect the leasl amount of prescripting. But internen 
can only act within the possibilities that have been established by the writing a 
programming. They may build simulated cities, Iry oul combat slralegies, tn a 
unique path through a labyrinthine web, or even prevent a murder, but urdi ihe 
imaginary world is nothing more than a costume trunk of empty avatars, of the 
interaclor's possible performances will have been called inlo being by i 
originaling author. 

Authorship in electronic media is procedural. Procedural authorship me; 
writing the rules by which the texis appear as well as writing the texis ihe 
selves. It means writing the rules for the interactor's involvement, ihal is. I 
condilions under which things will happen in response to ihe parlicipant's ; 
tions. It means eslablishing the properties of the objecls and polential objects 
the virtual world and the formulas for how they will reiale lo one another. 1 
procedural aulhor creales noljusl a set of scenes bul a world of narralive pos 
bililies. 

In electronic narrative the procedural author is like a choreographer \v 
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supplies the rhythms, the context, and the set of steps that will be performed. The 
interactor, whether as navigator, protagonist, explorer, or builder, makes use of 
this repertoire of possible steps and rhythms to improvise a particular dance 
among the many, many possible dances the author has enabled. We could per- 
haps say that the interactor is the author of a particular performance within an 
electronic story system, or the architect of a particular part of the virtual world, 
but we must distinguish this derivative authorship from the originating author- 
ship of the system itself. 

Interestingly enough, the question of authorship in formulaic media is one 
that students of ancient oral narrative have considered at length. In the 1930s, 
Greek scholars were distressed when literary analysis revealed that Homer (and 
other epic preliterate poets) created through a process that involved fitting stock 
phrases and formulaic narrative units together. Critics at that time resisted the 
thought that the great artist Homer was not original in the same way that mod- 
ern print-based writers are expected to be. Now, with the advent of computer- 
based authorship, we are experiencing the opposite confusion. Contemporary 
critics are attributing authorship to interactors because they do not understand 
the procedural basis of electronic composition. The interactor is not the author 
of the digital narrative, although the interactor can experience one of the most ex- 
citing aspects of artistic creation — the thrill of exerting power over enticing and 
plastic materials. This is not authorship but agency. 



Coda 

The Language oFthe 
Future 

by LaurieAnderson 



Last year, I was on a twin-engine plane coming 
from Milwaukee to New York City. Just over La 
Guardia, one of the engines conked out and we 
started to drop straight down, flipping over and 
over. Then the other engine died: and we went 
completely out of control. New York City started 
getting taller and taller. A voice carne over the 
intercom and said: 

Our pilot had informed us that we are about to 

attempt a crash landing. 

Picase extinguish all cigarettes. 

Place your tray tables in their upright, locked 

position. 

Your Captain says: Picase do not panic. 

Your Captain says: Place your head in your hands. 

Captain says: Place your head on your knees. 

Captain says: Put your hands on your head. 

Put your hands on your knees! 

(heh-heh) 

This is your Captain. 

Have you lost your dog? 

We are going down. We are all going down 

together. 

As it turned out, we were caught in a downdraft and 
rammed into a bank. It was, in short, a miracle. But 
afterwards I was terrified of getting onto planes. 
The moment I started walking down that aisle, my 
eyes would clamp shut and I would fall into a deep, 
impenetrable sleep. You don't want to see this. 
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You don't want to be 
here. Have you lost 
your dog? 

Finally I was able to remain conscious, but I always had 
to go up to the forward cabin and ask the stewardesses 
if I could sit next to them: "Hi!Uh,mindifljoinyou?" 
They were always rather irritated — "Oh, all right (what a 
baby)" — and I watched their uniforms crack as we 
made nervous chitchat. Sometimes even this didn't 
work, and I'd have to find one of the other passengers 
to talk to. You can spot these people immediately. 
There's one on every flight. Someone who's really on 
your wavelength. 

I was on a flight from L.A. when I spotted one of them, 
sitting across the aisle. A girl, about fifteen. And she 
had this stuffed rabbit set up on her tray table and she 
kept arranging and rearranging the rabbit and kind of 
waving to it: 

"Hi!" "Hi there!" And I decided: This is the one I want to 
sit next to. So I sat down and we started to talk and 
suddenly I realized she was speaking an entirely 
different language. Computerese. A kind of high-tech 
lingo. Everything was circuitry, electronics, switching. If 
she didn't understand something, it just "didn't sean." 

We talked mostly about her boyfriend. This guy was 
never in a bad mood. He was in a bad mode. Modey 
kind of guy. The romance was apparently kind of rocky 
and she kept saying: "Man oh man you know like it's so 
digital!" 

She just meant the relationship was on again, off again. 
Always two things switching. 
Current runs through bodies — And then it doesn't. 
It was a language of sounds, of noise, of switching, of 



signais. 

It was the language of the rabbit, the caribou, the 
penguin, the beaver. 
A language of the past. 

Current runs through bodies and then it doesn't. On 
again. Off again. 

Always two things switching. One thing instandy 
replaces another. It was 
the language of the future. 

PUT YOUR KNEES UP TO YOUR CHIN. 

HAVE YOU LOST YOUR DOG? 

PUT YOUR HANDS OVER YOUR EYES. 



JUMP OUT OF THE PLANE. 

THERE IS NO PILOT. YOU ARE NOT ALONE. 

THIS IS THE LANGUAGE OF THE ON-AGAIN 

OFF-AGAIN FUTURE. 

AND IT IS DIGITAL. 
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notes and references» 



ichard Wagner, "Outlines of the Artwork ofthe 
Future" 

3. The problem ofthe Theatrical edifice ofthe Future 
can in no wise be con-sidered as solved by our modern 
stage buildings: for they are laid out in ac-cord with 
traditional laws and canons which have nothing in 
common withthe requirements of pure Art. Where 
speculation for gain, on the one sidejoins forces with 
luxurious ostentation on the other, the absolute interests 
of Art must be cryingly affected; and thus no architect in 
the world will beable to raise our stratified and fenced- 
off auditoria — dictated by the par-celling of our public 
into the most diverse categories of class and civil sta- 
tion — to conformity with any law of beauty. If one 
imagines oneself, for amoment, within the walls ofthe 
common Theatre ofthe Future, one willrecognise with 
little trouble, that an undreamt width of field lies 
thereinopen for invention. — R. wagner. 

4. It can scarcely be indifferent to the modern 
landscape-painter to observeby how few his work is 
really understood to-day, and with what blear-eyed 
stupidity his nature-paintings are devoured by the 
Philistine world thatpays for them; how the so-called 
"charming prospect" is purchased to as-suage the 
idle, unintelligent, visual gluttony of those same 
need-less menwhose sense of hearing is tickled by 
our modern, empty music- 
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5. manufactureto that idiotic joy which is as repugnant a 
reward of his performance to theartist as it fully 
answers the intention of the artisan. Between the 
"charmingprospect" and the "pretty tune" of our 
modern times there subsists a dole-ful affinity, 
whose bond of union js certainly not the musing 
calm ofThought, but that vulgar slipshod 
sentimentality which draws back in selfish horror from 
the sight of human suffering jn its surroundings, to 
hire for it-self a private heavenlet in the blue mists of 
IMature's generality. These sen-timentals are willing 
enough to see and hear everything: only not the 
actual, undistorted Man, who lifts his warning finger on 
the threshold of their dreams.Butthisisthe very man 
whom we must setupintheforefrontof our show! — R. 
wagner. 

i. It is a little difficult to quite unravel this part of the 
metaphor, for the same word "Bode/?" is used twice 
over. I have thought it best to translate it in the 
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first place as "loam," and in the second as "ground"; for it appears as though 
the idea were, jn the former case, that of what agriculturists cali a "top- 
dressing," and thus a substance which could break up the lower soil and make 
jtfruitful.The "it" which occurs after the colon may refereither to the "feeling" 
or to the "orchestra," for both are neuter nouns. — tr. 4. The modern 
Playwright will feel little tempted to concede that Drama oug-: notto belong 
exclusively to Wsbranch of art,the art of Poesy;above all will he not be able to 
constrain himself to share jt with the Tone-poet, — to w!t, as he understands us ; 
allow the Play to be swailowed up by the Opera. Per-fectly correct! — so long as 
Opera subsists, the Play must also stand, and, for the matter of that, the 
Pantomime too; so long as any dispute hereon is thinkable, the Drama of the 
Future must itself remain un-thinkable. If, however, the Poet's doubt lie deeper, 
and consist jn this, that he cannot con-ceive how Song should be entitled to 
usurp entirely the place of spoken di-: alogue: then he must take for rejoinder, 
that jn two several regards he has not as yet a clear idea of the character of 
the Art-work of the Future. Firstly, he does not reflect that Music has to 
occupy a very different posi-tion jn this Art-work to what she takes in modern 
Opera: that only wherei her power is the fittest, has she to open out her ful I 
expanse; while, on the contrary, wherever another power, for instance that of 
dramatic Speech, js the most necessary, she has to subordinate herself to that; 
still, that Music possesses the peculiar faculty of, without entirely keeping 
silence, so im-perceptibly linking herself to the thought-full element of Speech 
that she lets the latter seem to walk abroad alone, the while she still supports 
jt. Should the poet acknowledge this, then he has to recognise jn the secor j 
place, that thoughts and situations to which the lightest and most re-strained 
accompaniment of Music should seem importunate and burden-some, can only 
be such as are borrowed from the spirit of our modern Play; which, from 
beginning to end, will find no inch of breathing-space within the Art-work of the 
Future.The Man who will portray himself in the Drama of the Future has done 
for ever with all the prosaic hurly-burly of fashionable manners or polite 
intrigue, which our modern "poets" have to tangle and to disentangle in their 
plays, with greatest circumstantiality. His nature-bidden action and hisspeech 
are:Yea, yea! and Nay, nay! — and all beyond js evil, Le., modern and 
superfluous. — R. wagner. 



Notes and References 405 



I.C.R. Licklider, "Man-ComputerSymbiosis" 

49. Webster's New International Dictionary, 2nd 
ed., G. and C. Merriam Co., Spring-field, Mass., 
1958, p. 2555. 

50. J. D. North, "The Rational Behavior of 
Mechanically Extended Man,"Boulton Paul 
Aircraft Ltd., Wolverhampton, Eng., September 
1954. 

51. H. Gelernter, "Realization of a 
GeometryTheorem Proving Machine," Unesco, 
NS, ICIP, 1.6.6, Internatl. Conf. on Information 
Processing, Paris, France, June 1959. 

52. A. Bernstein and M. deV. Roberts, "Computer 
versus Chess-Player," Scien-tlfic American, vol. 
198, June 1958, pp. 96-98. W.W. Bledsoe and I. 
Brown-ing, "Pattern Recognition and Reading 
by Machine," presented at theEastern Joint 
Computer Conf., Boston, Mass., December 1959. 
G. P. Din-neen, "Programming Pattern 
Recognition," Proc. WJCC, March 1955, pp.94- 
100. B. G. Farley and W. A. Clark, "Simulation of 
Self-Organizing Systems by Digital Computers," 
IRE Trans, on Information Theory, vol. IT-4, Sep 
tember 1954, pp. 76-84. R. M. Friedberg,"A 
Learning Machine: Part \,"IBMJ. Res&Dev., vol. 2, 
January 1958, pp. 2-13. P. C. Gilmore,"A Program 
for the Production of Proofs forTheorems 
Derivable Within the First OrderPredicate 
Calculus from Axioms," Unesco, NS, ICIP, 
1.6.14., Internatl. Conf. on Information 
Processing, Paris, France, June 1959. A. 
Newell, "The chess Machine: An 



53. Example of Dealing with a Complex Task by 
Adaptation," Proc. WJCC, March 1955, pp. 101- 
8. A. Newell and J. C.Shaw, "Programming the 
Logic Theory Machine," Proc. WJCC, March 

1957, pp. 230-44. A. Newell, J. C. Shaw, and H. A. 
Simon, "Chess-PlayingPrograms and the Problem 
of Complexity," IBMJ. Res & Dev., vol. 2, Octo-ber 

1958, pp. 320-35. 0. G. Selfridge, 
"Pandemonium, A Paradigm forLearning," Proc. 
Symp. Mechanisation of Thought Processes, 
Nati. Physical Lab.,Teddington, Eng., November 
1958. C. E. Shannon, "Programming a Computer 
for Playing Chess," Phil. Mag., vol. 41, March 
1950, pp. 256-75. H.Sherman,"A Quasi- 
Topological Method for Recognition of Line 
Patterns/'Unesco, NS, ICIP, H.L.5, Internatl. Conf. 
on Information Processing, Paris, France, June 
1959. 

54. A. Newell, J. C. Shaw, and H. A. Simon, 
"Report on a General Problem-Solving Program," 
Unesco, NS, ICIP, 1.6.8, Internatl. Conf. On 
Information Processing, Paris, France, June 
1959. 



406 Notes and KeFerences 

Douglas Engelbart, "Augmenting Human 

Intellect: A Conceptual Framework" 

30. Kennedy and Putt bring out the importance of 
a conceptual frameworkthe process of 
research. They point out that new, multi- 
disciplinarysearch generally finds no such 
framework to fit within, that a framewcof sorts 
would grow eventually, but that an explicit 
framework-searphase preceding the research js 
much to be preferred. J. L. Kennedy andH. Putt, 
"Administration of Research in a Research 
Corporation/' RACorporation Report P-84 7 (April 
20,1956). 

3 1 . Ross Ashby, Design for a Brain, New York: 
John Wiley & Sons, 1960. R<Ashby, "Design for 
an Intelligence-Amplifier," in Automata Studies, 
C.Shannon and J. McCarthy, ed., Princeton, N.J.: 
Princeton University Pre1956, pp. 215-34. 



Roy Ascott, "Behaviourist Art and Cybernetic Vision" 

13. J. Diebold, "The Economic and Social Effects 
of Automation," Proc. 2ndternational Congress on 
Cybernetics, International Association for 
CybernetiNamur, 1958. F. H. George, The Brain 
Asa Computer, London: PergamPress, 1961. 

14. C. Chandessais, "Essai sur la formalisation 
dans les sciences du coportement," 
Cybernetica, Journal of the International 



15. Association for CybernetNamur, 1964, no. 1. 

16. W. Cordon, "Economic and Social Effects of 
Automation," Cybemetica, Jenal ofthe International 
Association for Cybernetics, Namur, 1964, no. 4. 

17. M. McLuhan, Understanding Media, London: 
Routledge and Kegan Pa1964. 

18. J. Bronowski, Science and Human Values, 
London: Hutchinson, 1961. 

19. L. Couffignal, "Que peut apporter la 
Cybernetique a la pedagogie," Cytnetica, Journal 
ofthe International Association for Cybernetics, 
Namur, 1964, no.: 

20. P. de Latil, La penseeartifldelle, Paris: 
Gallimard, 1952. 

21. W. R. Ashby, "Design for an Intelligence 
Amplifier," in E. Shannon and McCarthy, eds., 
Automata Studies, Princeton, N.J.: Princeton 
UniversPress, 1956. 

22. A. G. Pask,"The Growth Process inside the 
Cybernetic Machine," Proc. 



Notes and Referentes 407 



Congress on Cybernetics, International Association 

for Cybernetics, Namur, 

1958. 

George, op. cit. 



n Kay, "User Interface: A Personal View" 

Bruner, Jerome. Towards a Theory of Instruction. New 
York: W. W. Norton and 
Company, 1966. 

Dawkins, Richard. The Blind Watchmaker. New York: 
Penguin Books, 1986. 

GallweyTim. The Inner Game of Tennis. New York: 
Random House, 1974. 

Haber, R. "How We Remember What We See," 
Sdentific American, vol. 222, 
1970, pp. 104-12. 

Hadamard, Jacques. An Essay on the Psychology 

of Invention in the Mathematical 

Field. New York: Dover, 1945. 

McLuhan, Marshall. Understanding Media: The 

Extensions of Man. New York: 

McGraw-Hili, 1964. 

Minsky, Marvin, and Seymour Papert. Perceptrons: 

An Introduction to Computa- 

tionalGeometry. Cambridge, Mass.: MIT Press, 1969. 

Mumford, Lewis. The Mythofthe Machine. 2 vols. New 

York: Harcourt, Brace 

and World, 1967-70. 

. Technics and Civilization. New York: Harcourt, 
Brace and Company, 
1934. 



Piaget, Jean. Judgment and Reasoning \n the Child. 
New York: Harcourt, Brace 
and Company, 1928. 

.- The Language and Thought ofthe Child. New 
York: Harcourt, Brace and 
Company, 1926. 

. The Origins of Intelligence in Children. New 
York: International Universi- 
ties Press, 1952. 

Suzuki, Shinichi. Nurtured by Love: A New 
Approach to Education. New York: Ex- 
position Press, 1969. 



Nelson, ComputerLib/Dream Machines 

See T. H. Nelson, The Snunking ofthe Heart: On the 
Psychology of Puns and Preterism in Carro// and 
Others, 1980, unless a decent writing system comes 
along. 

408 Notes and Referentes 



Alan Kay and Adele Goldberg, "Personal Dynamic 
Media" 

55. Baeker, Ronald. "A Conversational Extensible 
System for the AnimationShaded Images," Proc. 
ACM SIGGRAPH Symposium, Philadelphia, Pa., 
Ju-1976. 

56. Goldberg, Adele, and Alan Kay, eds. Smalltalk- 
72lnstruction Manual, Xerox PaAlto Research 
Center, Technical Report No. SSL 76-6, March 
1976. 

57. Goldeen, Marian. "Learning About Smalltalk," 
Creative Computing, Septenber-October 1975. 

58. Kay, Alan. The Reactive Engine. Doctoral 
dissertation, University of UtatSeptember 1969. 

59. Learning .Research Group. "Personal 
Dynamic Media," Xerox Palo AltResearch 
Center, Technical Report No. SSL 76-1, March 
1976. 

60. Saunders, S. "Improved FM Audio Synthesis 
Methods for Realtime Digtal Music Generation," 
Proc, ACM Computer Science Conference, 
WashingtorD.C, February 1975. 

61. Smith, David C. PYGMALION: A Creative 
Programming Environment, Doctore 
dissertation, Stanford University Computer 
Science Department, Jun1975. 

62. Snook,Tod. Three-dimensionalGeometric 
Modelling. Master'sthesis, University oCalifornia, 
Berkeley, September 1976. 



Tim Berners-Lee, "Information Management: A 
Proposal" 

32. T. H. Nelson, "Getting It Out of Our System," jn 
Information Retrieval: f-Critical Review, 
G.Schechter, ed., Washington, D.C. Thomson 
Books, 1967pp. 191-210. J. B. Smish and S. F. 
Weiss, "An Overview of Hypertext," ir 
Communications oftheACM, vol. 31, no. 7, July 
1988, and other articles irthe same special 
"Hypertext" issue. B. Campbell and J. Goodman, 
"HAM:A General Purpose Hypertext Abstract 
Machine," in Communications otheACM, vol. 
31, no. 7, July 1988. R. M. Akscyn, D. 
McCracken, anaE. A. Yoder, "KMS: A 
Distributed Hypermedia System for Managinc 
Knowledge in Originations," in Communications 
oftheACM, vol. 31, no. 7, July1988. 

33. Hypertext on Hypertext, a hypertext version of the 
special Comms of the ACMedition, is available 
from the ACM for the Macintosh or PC. 

34. J. Moline et al. eds. Proceedings ofthe 
Hypertext Standardisation Workshop, Januar. 



Notes and Kererences 409 



16-28, 1990, National Institute of Standards and 
Technology, pub. U.S. Dept. of Commerce. 

63. Under unix, type man rn to find out about the 
rn command which is usedfor reading uucp news. 

64. Under VMS, type HELP NOTES to find out 
about the VAX/NOTES system. 

65. On CERI\IVM,type FIND DOCFIND for 
information about howto accessthe CERNDOC 
programs. 



ieorge Landow and Paul Delany, "Hypertext, 
Hypermedia nd Literary Studies: The State ofthe 
Art" 

35. Jay David Bolter, "Topographic Writing: 
Hypertext and the ElectronicWriting Space," jn 
Hypermedia and Literary Studies, George Landow 
and PaulDelany, eds., Cambridge, Mass.: MIT 
Press, 1991. 

36. The term Hypertext was coined byTheodor H. 
Nelson in the 1960s. 

37. Roland Barthes, S/Z, trans. Richard Miller, 
New York: Hill &Wang, 1974. We have general ly 
used the term blocks here, though its connotations 
are un-fortunate, given the actual fluidity and 
readiness to bond of electronictexts. See also 
discussion of nodes in John Slatin, "Reading 
HypertexLOrder and Coherence jn a New 
Medium," jn Landow and Delaney, op. cit. 

38. Vannevar Bush, "As We May Think," Atlantic 
Monthly no. 176, July 1945: pp.101-8. James M. 
Nyce and Paul Kahn have shown that Bush had 
writtena longer version of this essay by 1937: 
"Innovation, Pragmatism, and Tech-nological 
Continuity: Vannevar Bush's Memex," Journal ofthe 
American Asso-ciation for Information Science, no. 
40,1989, pp. 214-20. 



39. In oral cultures, of course, the text had quite a 
different status jn the mind, one that was closer in 
some respects to the hypertextual model. See 
Wal-ter J. Ong, Oralityand Literacy: The 
Technologizing ofthe Word, London: Methuen,1982. 

40. Lyrical poems are a special case: they are 
independent works, but have aciose intertextual 
relationship with others in the same tradition, or 
by thesame author. 

41. We may here distinguish between hypertext 
and a related concept, Standard Generalized 
Mark-up Language (SGML). SGML concerns the 
mark-ing out of units in an electronic text and then 
attaching to them a "tag" ordescriptive category 
(such as "noun phrase" or "animal image"). A 
pro- 



410 Notes and References 

gram can then sort the tags to support 
grammatical or stylistic analysi; SGML thus sorts 
textual units by categories; hypertext links the units 
int larger structures. 

66. Nicole Yankelovich, Norman Meyrowitz, and 
Andries van Dam, "Readinand Writing the 
Electronic Book," in Landow and Delany op. cit. 

67 . For some speculations on these textual 
megastructures or "docuverses,seethediscussion of 
the "New Alexandria" in the original, unedited versio 
of thisessay in Landow and Delaney, op.cit., and 
Yankelovich, "From Electronic Books to Electronic 
Libraries," in Landow and Delaney, op. cit. 



42. A notable proto-hypertextual work is Humphrey 
Jennings, PandemoniumThe Corning ofthe 
Machine as Seen by Contemporary Observers, 
1660-1886, New YorkFree Press, 1985.This js a 
chronological series of short passages and jmages 
on the Industrial Revolution; Jennings's posthumous 
editor, CharleMadge, usted sixteen "Theme 
Sequences," each of them an alternate ordering of 
groups of passages. Jennings's remarkable work 
could very easily and usefully be adapted for 
hypertext presentation. 

43. Examples of educational hypertext include Ben 
Schneiderman's HypertieHolocaust materials, the 
many sets of HyperCard materials, including trn 
Harvard Perseus Project (see Gregory Crane and 
Elli Mylonas, "Anden Materials, Modern Media: 
Shaping the Study of Classics with Hypertext/in 
Landow and Delany, op. cit.), and the Intermedia 
materials developed a Brown. Among the 
applications of hypertext to poetry and fiction one 
cainumber Michael Joyce's, "Afternoon," Stuart 
Moulthrop's adaptation oBorges's "Forking 
Paths"to both HyperCard and StorySpace (see 
StuarMoulthrop, "Reading from the Map: Metonymy 
and Metaphor in the Fietion of "Forking Paths,' " in 
Landow and Delany, op. cit.), and Willian Dickey's 
mixed media poetry in HyperCard (seeWilliam 
Dickey, u Poem Descending a Staircase: Hypertext 
and the Simultaneity of Experience," NLandow and 
Delany, op. cit.). 

44. Kenneth Morrell, "Teaching with HyperCard. An 
Evaluation ofthe ComputerBased Section in 
Literature and Arts C-14:The Concept ofthe Hero in 
Hel-lenic Civilization." Perseus Project Working 
Paper 3. Cambridge, Mass. Department of Classics, 
Harvard University, 1988. 



45. Marshall McLuhan, The Gutenberg Galaxy: The 
Making of Typographic Man, Toronto: University of 
Toronto Press, 1962; Elizabeth L. Eisenstein, Tht 
Printing Press as an Agent of Change: 
Communications and Cultural Transformations ii 
Early-Modern Europe, 2 vols. Cambridge, Mass.: 
Cambridge University Press, 1979; and J. David 
Bolter, Writing Space: The Computer, Hypertext 
and the Histor] 



ofWrlting Hillsdale, N.J.: Lawrence Erlbaum, 1990.We 
would Nketothank Professor Bolter for sharing a draft of his 
work with us before publication. 



lorton Heilig, "The Cinema oFthe Future" 

68. The nude eye actually has a vertical range of 180°, 
but this js reduced toapproximately 150° by the brow 
and cheek of the head. 

69. Vistavision, by photographing first on a negative 
frame twice the originalsize before printing on to normal 
size positive, has partially returned screen image to its 
usual sharpness. 



Sutherland, "The Ultimate Display" 

46. K.C. Knowlton."A Computer Technique for 
Producing Animated Movies," Proceedings ofthe 
SpringJoint Computer Conference Washington, D.C.: 
Spartan, 1964. 

47. I. E. Sutherland. "Sketchpad — A Man-Machine 
Graphical Communica-tion System," Proceedings of 
the SpringJoint Computer Conference. Detroit, Mich. 
May 1963 Washington, D.C.: Spartan, 1964. 



cott Fisher, "Virtual Interface Environments" 

ates 

23. The principal projectteam at NASA js C. Coler, S. 
Fisher, M. McGreevy, W.Robinett,and E.Wenzel. At 
Sterling Software: S. Bryson,J. Humphries, R.Jacoby, 
D. Kaiser, D. Kerr, and P. Stone. 

24. The principal project team at VPL was T. 
Zimmerman, C. Blanchard, S.Bryson, and J. Grimaud. 

iferences 

1. Comeau, C v and J. Bryan. "HeadsightTelevision System 
Provides Remote Surveillance." In Electronics, 
November 10,1961, pp. 86-90. 

19. Fisher, Scott S. "Telepresence Master Glove 
Controller for Dexterous R;botic End-Effectors." In 
Advances in Intelligent Robotics Systems, D. P. 
Casasered. Proc. SPIE 726,1986. 

20. Fisher, S. S., M. McGreevy, J. Humphries, and W. 
Robinett. "Virtual Envronment Display System." ACM 
1986 Workshop on 3D Interactive Graptles, Chapel 
Hill, l\I.C. October 23-24, 1986. 

21. Fisher, S. S., M. McGreevy, J. Humphries, and W. 
Robinett. "Virtual lnte 1 face Environment for 
Telepresence Applications." In Proceedings ofANS '. 
ternational Topical Meeting on Remote Systems and 
Robotics in Hostile Environments.. D. Berger, ed., 
1987. 

22. Fisher, S. S., E. M. Wenzel, C. Coler, and M. W. 
McGreevy. "Virtual Inteface Environment 
Workstations." In Proceedings ofthe Human Factors 
Sor? '32no Annual Meeting, Anaheim, Calif., October 
24-28,1988. 

23. Foiey, James D. "Interfaces for Advanced 
Computing," Scientific Ameravo\. 257, no. 4 (1987), 
pp. 126-35. 



24. Herot, C. "Spatial Management of Data." ACM 
Transactions on Database S.. terns, vol. 5, no. 4 
(1980). 

25. Knowlton, K. C. "Computer Displays Optically 
Superimposed on Input D-. vices." The Bell System 
Technical Journal, vol. 56, no. 3 (March, 1977), p: 
367-83. 

26. Lippman, Andrew. "Movie-Maps: An Application of 
the Optical Videodisto Computer Graphics." 
Computer Graphics, vol. 14, no. 3 (1980), pp. 32f A . 

14.Negroponte, N. "Media Room." Proceedings ofthe 
Society for Information D/'sp/vol. 22, no. 2 (1981), 
pp. 109-13. 

15.Schmandt, C. "Spatial Input/Display 

Correspondence in a StereoscopComputer Graphic 
Work Station." Computer Graphics, Proceedings of 
ACM S.". GRAPH '83, vol. 17, no. 3 (1983), pp. 253ff. 

16. Sutherland, I. E. "Head-Mounted Three-Dimensional 
Display." Proceedr.ofthe FallJoint Computer 
Conference, vol. 33 (1968), pp. 757-64. 

17.Wenzel, E. M., F. L. Wightman, and S. H. Foster. "A 
Virtual Display Syitem for Conveying Three- 
Dimensional Information." Proc. Hum. Fac. So 
(1988). 

jarcos Novak, "Liquid Architectures in Cyberspace" 

otes 

5. Bruce Sterling has recently collected five: Gibsonian 
cyberspace, Barlov-jan cyberspace, Virtual Reality, 
Simulation, andTelepresence.The definitionabove is a 
concatenation of all five plus a sixth, set forth in 
"Making Reality a Cyberspace" by Wendy Kellog 
and others. 



6. Thomas Kuhn discussed the notion of world lines 
with respect to scientificconcepts in atalk on 



untranslatability between different intellectual frame- 
works (Kuhn, 1990). 



V 



Ferences 

70. Alberti, L. B. [1404-14721 . The Ten Books of 
Architecture. New York: DoverPublications, 1986. 

71. Anderson, J. A., and E. Rosenthal, eds. 
Neurocomputing: Foundations of Research. Cambridge, 
Mass.: MIT Press, 1988. 

72. Benedikt, M., ed. Cyberspace: Collected Abstracts. 
University of Texas at Austin, 1 990 . 

\. Berman, M. Corning to Our Senses. New York: Simon 
and Schuster, 1989. j. Bonner, J. T. The Evolution of 
Complexity by Means of Natural Selection. Princeton, 

N.J.: Princeton University Press, 1988. i. Brisson, 
D. W., ed. Hypergraphics: Visualizing Complex 
Relationships in Art, Science and 

Technology. Boulder, Colo.: Westview, 1978. . 
Conrads, U. Programs and Manifestoes on 20th- 
century Architecture. Cambridge, 

Mass.: MIT Press, 1970. . Coyne, R. D. u Toolsfor 
Exploring Associative Reasoning jn Design." In The 

Electronic Design Studio, M. McCullough, W. J. 

Mitchell, and P. Purcell, eds. 

Cambridge, Mass.: MIT Press, 1990. 

Davies. P., ed. The New.Physics. Cambridge, U.K.: 

Cambridge University 

Press, 1989. 



Dawkins, R. The Blind Watchmaker. New York: W. 

W. Norton & Company, 

1986. 



Feyerabend, P. Against Method. London: Verso, 1988. 
Gregory, B. Inventing Reality: Physics As 
Language. New York: Wiley Science 
Editions, 1988. 



414 Notes and References 

3. Helsel, S. K., and J. P. Roth, eds. Virtual Realities: 
Theory, Practiceand FLondon: Meckler, 1991. 

4. Hillier, B., and J. Hanson. The Social Logic of 
Space. Cambridge, U.K.: '.bridge University Press, 
1984. 

5. Hollier, D.AgainstArchltecture: The Writingsof George 
Bataille. CambridgeMIT Press, 1989. 

6. Kostoff, S. A Hi storyof Architecture. Oxford: Oxford 
University Press, 19: 

7. Kuhn,T. "Untranslatabilty," lecture notes, UCLA, 
1990. 

8. Kuppers, B.O. Information and the Originof Ufe. 
Cambridge, Mass.: MIT 1990. 

9. Langton, C. G. Artificial Life: Proceedings ofan 
Interdisdplinary Workshop on.thesis and Simulation of 
Living Systems Held September 1987. Reading, tfz. 
Addison-Wesley, 1989. 

10. Lorca, F. G. Poetin New York. New York: Noonday 
Press, 1989. 

11. McClelland, J. L,and D. E. Rumelhart,eds. 
ParallelDistrlbutedProcess-.plorationinthe 
Microstructure of Cognition, vols. 1 and 2. 
Cambridge, Mass. Press, 1986. 

12. Negroponte, N. Soft Architecture Machines. 
Cambridge, Mass.: MIT 1975. 



13. Perez-Gomez, A. Architecture and the Crisis of Modern 
Science. CambridgeMIT Press, 1983. 

14. Pickover, C. Computen, Pattern, Chaos and Beauty: 
Graphics from an Unsee r New York: St. Martin's 
Press, 1990. 

15. Sartre, J. P. Being and Nothingness, New York: 

Washington Square -1956. 
16. Scarry, E. The Body \n Pain. New York: Oxford 
University Press, 1985. 

17. Sterling, B. "Cyberspace (TM), "Interzone, 
November 1990. 

18. Tzonis, A. Classical Architecture: The Poetics of 
Order. Cambridge, Mass. Press, 1986. 

19. Vidler, A. "The Building in Pain." In AA Files. 
London: AA Press, 199; 

Woods, L. OneFiveFour. New York: Princeton Architectural Press, 198= 



Char Davies, "Changing Space: Virtual 
Reality as an Arena of Embodied 
Being" 

This paper was presented in abridged form at 
Consciousness Reframed, 1st C Research Conference, 
Centre for Advanced Inquiry jn the Interactive Arts, . 

i/ersity of Wales College, Newport,Wales, July 1997, and at Beyond ShelterThe 
Future of Architecture,The Graham Foundation, Chicago, III., September 1997. It 
is partially based on the presentations "Soul jn the Machine: Osmose — The 
Paradox of Being in Immersive Virtual Space" and "Osmose as Metaphor: Alter- 
native Aesthetics and Interaction in Immersive Virtual Space," delivered at the 
annual meeting of the Association for Computing Machinery (ACM) Special In- 
terest Group in Graphics, New Orleans, La., August 1996. 
Osmose was designed as an alternative to the dominant aesthetic and inter-active 
sensibility of virtual reality. The work was created by myself, John Harrison, and 
Georges Mauro, with sound by Rick Bidlack and Dorota Blaszczak, and was 
produced by Softimage between 1994 and 1995. Public installations of the work 
were made in an intimate enclosed immersionarea with a darkened visitor space. 
Visitors were able to "witness" each jm-mersive journey from the immersant's 
point of view as it took place via liveaudiovisual connection. Adjacent to the video 
projection was a live shadowprojection of the jmmersant, providing an 
associative link between his orher body as conduit for lived experience, and the 
work's consequent jm-agery and sound. For a more detailed description, see 
"Osmose: Notes onBeing in Immersive Virtual Space," paper delivered at the 
6th International Symposium on Electronic Art, Montreal, September 1995, and 
CharDavies and John Harrison, "Osmose — Towards Broadening the Aesthetics of 
Virtual Reality," in ACM Computer Graphics, vol. 30, no. 4, November 1996, 



73. pp. 25-8. 

74. The subjective experience of being spatially 
enveloped or immersed in avirtual environment is key 



to my work and the views expressed jn thispaper. 
By "jmmersion" or "jmmersive virtual space" I 
specifically meanimmersion within a spherically 
360-degree, totally enveloping virtualspace, 
implying a "being within." In my experience with VR, 
such sensa-tion of envelopment is possible only 
through the wearing of a head-mounted display 
helmet with a wide field of view. While the word 
"immersive" is currently being used by the industry 
to describe wrap-around screens and domes 
(creating what I consider to be nonimmersive 
experiences) my use of the word "jmmersive" denotes 
a totally envelopingvirtual space. 
75. Gaston Bachelard, The Poeticsof Space (Boston, 
Mass.: Beacon Press, 1966, p. 206. For Bachelard's 
discussion on the psychologically transforming 
qualities of immense open spaces such as the sea, 
desert, and plains, see pp.203-10. 
nui tb onu nei ti 

48. Arthur Deikman, "Deautomatization and Mystical 
Experience" and "Experimental Meditation," jn Altered 
States of Consciousness, Charles Tart, ed., New York: 
HarperCollins, 1990, pp. 50, 262-3. Note that jn terms 
of "re-ceptivity" to Osmose, I have been told by at 
least one individual that al-though she was 
intellectually skeptical when she went jn, "something 
happened" and much to her surprise she became 
entranced This points tothe medium's potency, for 
better and worse. 

49. Deikman, "Deautomatization," p. 52. 

50. Ibid., pp. 47-55. 



51. Y. Karim, personal correspondence with the author, 
September 1 995. "Osmose heightened an awareness of 
my body as a site of consciousness and ofthe experience 
and sensation of consciousness occupying space." 



52. This method was partially informed by my own 

practice of scuba diving j r the deep sea, certainly a 

"psychically innovating space" as Bachelard sug- 
gested. Diving at depths of 200 feet over a 6,000-foot 

abyss introduced meto the experience of being within 

an almost pure, abstract yet sensuous'.. enveloping 

space, where when there js nothing to "look at" 

perceptual andcognitive distinctions between near and 

far, inside and out, really do dis-solve. While diving, 

navigation and buoyancy are achieved through subtle 

and skillful use of breath and balance, and the use of 

hands js discouraged.There are other relevant 

comparisons between diving and immersive virtual 

environments, such as the donning of heavy gear in 

order to access suc"spaces, as well as the necessity of 

limiting the length of the experience inorder to avoid 

possible dangers to one's health. 
Verana Conley, preface to \/(1993), in Rethinking Technologies, Verana Conley, ed., 
Minneapolis: University of Minnesota Press, 1993, p. ix. 



76. Ibid., p. xii. 

In its most prevalent form, immersive virtual reality can be considered tobe "a 
literal re-enactment of Cartesian ontology." Richard Coyne, "Hei-degger & 
Virtual Reality: The Implications of Heidegger's Thinking forComputer 
Representaron," Leonardo, vol. 27, no. 1, 1994, pp. 66-75. Interms of visuals, 
most real-time three-dimensional computer graphic tech-niques are based on 
representing "hard-edged solid objects in empt.. space" — in a combination of 
low-level mimetic realism, Cartesian space, and Renaissance perspective. In 
virtual environments the human subject jsusually reconstructed as an omnipotent 
and isolated viewpoint maneuver-ing in empty space and probing objects with an 
acquisitive hand. Interac- 



inotbs ana Kererences 417 



tivity in many commercial computer games involves 
adrenaline-producing high-speed action and aggression. 
These approaches tend to reinforce a Cartesian way of 
seeing the world in terms of emphasizing the separation 



of subject over object, mind over body, and the world as 
"standing-reserve." For a discussion about the world 
being reduced to "standing-reserve" for human 
consumption, see Martin Heidegger, "The Question 
Concerning Technology," jn The Question Concerning 
Technology & Other Essays, New York: Harper & Row, 
1977, pp. 17-27. The imperative to master and control 
is not sur-prising given the technology's origins, not only 
jn value-wise jn the Western philosophic tradition, but 
instrumentally in the military as well. 



II Viola, "Will There Be Condominiums in Data 
Space?" 

'.. A. R. Luria, The Mind of 'the Mnemonist, New York: Basic 
Books, 1968. 

!. The Greeks perfected a system of memory that used 
the mental jm-printing of any objects or key points to 
be remembered onto specific lo-cations along a 
pathway previously memorized from an actual temple. 
To recall the points jn their proper order, one simply had 
to take the walk through the temple in one's mind, 
observing the contents left at each lo-cation along the 
way. 

. A. K. Coomaraswamy, The Transformaron of Nature in Art, 
NewYok: Dover Pub-lications, 1956; reprint of the 
original Harvard University Press edition, Boston, 1934. 
A Landmark Interactive laserdisc project by MIT Media 
Lab, in the late 1970s, that mapped the city of Aspen, 
street by street, with moving cameras so that the viewer 
could take a "ride" through the city, going anywhere at will 
— one of the first visual-mapping database moving- 
image projects related to data space jdeas and today's 
virtual reality technology. 



Ascott, "Is There Love in the Telematic Embrace? 



The neologism "telematique" was coined by Simon Nora and Alain Mine in 
L'lnformatisation de la societe, Paris: La Documentation Francaise, 1978. The 
German word "Gesamtkunstwerk" was used by Richard Wagnerto refer to his 
vision of a "total artwork" integrating music, image, and poetry. 



418 Notes and ReFerences 

77. Humberto R. Maturana and Francisco J. Varela, 
The TreeofKnowledge: Thelological Roots of Human 
Understanding, Boston: Shambhala, 1987. 

78. Roland Barthes, "From Work to Text," jn Image- 
Music-Text, trans. StepNHeath, New York: Hill 
&Wang, 1977. 

79. C. E. Shannon and W. Weaver, The Mathematical 
Theory of Communication, Jbaha: University of 
Illinois Press, 1949. 

80. Heinz von Foerster, Observing Systems, New 
York: Intersystems, 1981. 

81. James Gleick, Chaos, New York: Heinneman, 
1987. 

82. J. A. Wheeler and W. H. Zurek, Quantum Theory 
and Measurement, PrincetoiN.J.: Princeton 
University Press, 1983. 

83. See Michel Sanouillet, ed., Salt Seller: The 
Writings of Marcel Duchamp (Manhand du Sel) New 
York: Oxford University Press, 1973. 

53. Charles Fourier (1772-1837). His "system of 
passionate attraction (elaborated jn 
Theoriedesquatresmouvementset des destinees 
generates C Pahs: Bireaux de la Phalange, 1841]) 
sought universal harmony. 

25. Peter Russell, The Awakening Earth, London: 

Routledge, 1982. 



27. James L. McClelland, David E. Rummelhart, 
and the PDP ResearcGroup, Parallel Distributed 
Processing, vol. 1, Cambridge, Mass.: MIT Presj 
1986. 

28. The term was first proposed in Roy Ascott, "Art 
and Telematics: Towarda Network Consciousness," 
in Art Telecommunications. Heidi Grundmann, ed 
Vancouver: Western Front, 1984. 

18. Paul J. Curran, Principles of Remote Sensing, New 
York: Longman, 1985. 

7. Stephen R. Graubard, ed., The Artificial Intelligence 
Debate, Cambridge, Mass. MIT Press, 1988. 

8. Koji Kobayashi, Computers and Communications, 
Cambridge, Mass.: MIT Pres!1986. 

9. Roy Ascott, "Art, Technology and Computer 
Science," in XLII Esposizionlnternazionale d'Arte 
(Venice: Biennale di Venezia, 1986). The interna 
tional commissioners for the 1986 Biennale were 
Roy Ascott, Don ForesteTom Sherman, and 
Tomaso Trini. 

10. Roy Ascott, "Gesamtdatenwerk: Konnektivitat, 
Transformation and Trar]szendenz," in 
KunstforumlQ3 September/October 1989. Project 
by Roy Ascott in collaboration with Peter 
Appleton, Mathias Fuchs, RoberPepperell, 
and Miles Visman. 

1 1 . Norbert Wiener, Cybernetics or Control and 
Communication in the Animal and MachlnCa m b ri d g e , 
Mass.: MIT Press, 1948. 



tiererences 419 



20. VPL Research, lnc v of California demonstrated 
"shared virtual reality" and"walk-through 
cyberspace"atTexpo '89 jn San Francisco, June 1989. 



'avel Curtfs, "Mudding: Social Phenomena in Text- 
Based 'irtual Realities" 

84. Forster, E. M., "The Machine Stops," in The Science 
Fiction Hall of Fame, vol. MB, Ben Bova, ed., Avon, 1973. 
Originally jn E. M. Forster, The Eternal Mo-ment and 
Other Stories, New York: Harcourt Brace Jovanovich, 
1928. 

85 . Eric S. Raymond, ed., The New Hacker's Dictionary, 
Cambridge, Mass.: MITPress, 1991. 

86. In fact, these two commands are so frequently used 
that single-characterabbreviations are provided for 
them The two example commands wouldusually be 
typed as follows: 

Can anyone 
hear me? 
:smiles. 

4. Forster, op cit. 

5. The "MOO" in "LambdaMOO" stands for "MUD, 
Object-Oriented." The 

origin of the "Lambda" part is more obscure, based on 
my years of expe- 
rience with the Lisp programming language, b. Van 

Gelder, Lindsy, "The Strange Case of the Electronic 

Lover," in Dunlop 



and Kling, op. cit. 7. Sara Kiesler et al., "Social 
Psychological Aspects of Computer-Mediated 

Communication," in Computerization and Controversy, 
Charles Dunlop and 

Robert Kling, eds., San Diego, Calif.: Academic Press, 
1991. Kiesler and 

her colleagues nave investigated the effects of 
electronic anonymity on the 
decision-making and problem-solving processes in 
organizations; some of 

their observations parallel mine given here. '. Van 
Gelder, op cit. . Morningstar, Chip, and F. Randall 
Farmer, "The Lessons of Lucasfilm's 

Habitat," in Cyberspace, Michael Benedikt, ed., 

Cambridge, Mass.: MIT 

Press, 1991. 
. Forster, op 
cit. 

420 Notes and Referentes 

54. McLuhan, Marshall, Understanding Media, New 
York: McGraw-Hill, 19i 

55. Forster, op cit. 



Fierre Levy, "The Art and Architecture oF 
Cyberspace" 

1. William Gibson, Neummancer, New York: Ace 
Science Fiction Books,] 



Janet Murray, "Agency" 

26. Gilies Deleuze and Felix Guattari, A Thousand 
Plateaus: Capitalism and ;phreriia, Minneapolis: 
University of Minnesota Press, 1987. 

27. Stuart Moulthrop, "Containing Multitudes:The 
Problem of Closureteractive Fiction," Association 
for Computen and Humanities Newsletter, vol 988, 
p. 1. 

28. The authoring environment Storyspace, 
distributed by Eastgate Syswas designed by Jay 
David Bolter, John Smith, and Michael Joyce. the 
virtue of allowing the writer to see a text as a series 
of lexias dispas boxes with links between them in 
the form of arrows. Storyspace pan important role jn 
popularizing hypertext writing in academic circlfore 
the arrival of the World Wide Web. 

29. The robbery story is "Flow" by Robert Frederick. 

30. David Thorburn, "Television Melodrama," jn 
Television: The Critical VieVace Newcomb, ed., 
Oxford, England: Oxford University Press, pp. 53 

31. I am grateful to Matthew Gray for telling me 
about this puzzle in Z 

32. Of course, as the reader response critics (see 
Norman N. H ol land, namics of Literary Response, 
Oxford, England: Oxford University Press,Norman N. 
Holland, 5 Readers Reading, New Haven, Conn.: 
Yale Unj\Press, 1991; Wolfgang I ser, The Act of 
Reading: A Theory of Aesthetic ReBaltimore, Md.: 
Johns Hopkins University Press, 1978; Umberto Ei 
Role of the Reader, Bloomington: Indiana University 
Press, 1979) remireading and viewing are not passive 
experiences at al I but require us tstruct the story jn 
an active manner.There is still a difference, howev 
tween this emotional and cognitive activity and the 
external actions v\jn a game or electronic narrative. 
Perhaps the most important diffijs that jn the latter 
we are conscious of our activity, which changes c 
lation to the story. 



29. For more on the interpretaron of games, jn a more 
anthropological ratherthan literary context, see Elliot 
M. Avedon and Brian Sutton-Smith, TheStudyof 
Games, Huntington, N.Y.: R. E. Krieger, 1979. 

30. Tetris (Spectrum Holobyte, 1987) was originally 
designed by AlexeyPazhitnov in Moscow jn 
1985.Thanks to Elizabeth Murray and HenryJenkins 
for sharing their Tetris experiences with me. 

.0. For a consideration of the agon as a characteristic 
organizing form of oral culture, see Walter J. Ong, 
Oralityand literacy: The Technologizing ofthe Word, 
New York: Methuen, 1982, pp. 43-45. 



422 

a note on the web site 



artmuseum.net http://www.artmuseum.net/w2vr 



Multimedia: From Wagnerto Virtual Reality \s a unique hybrid pu:-lishing project 
that joins W.W. Norton & Company with o_-media partner, Intel Corporation's 
ArtMuseum.net, to pre-sent an untold history of multimedia. 

The book and the Web site are meant to work in ta."-dem. On-line, 
Multimedia: From Wagnerto Virtual Reality is a :.-namic, growing resource 
featuring hyperlinked texts ana wealth of multimedia documentaron — vintage 
photogra:": rare video clips, and an Interactive timeline chronicling r medium's 
pioneers. It also includes "in-depth" profiles d several of the artists and 
scientists featured in the book A -tending its scope to include illustrated 
descriptions oft artworks and technological invention that underlie the -. : lution 
of the medium. 

The site was developed jn ciose collaboration with A Museum.net and Intel's 
Arts and Education program. '.: our hope that the Web site will serve as a 
valuable res: for students, teachers, artists, and critics probing the active media 
arts, as well as those attempting to make s=-:-of this burgeoning medium that is 
transforming our / our culture. 
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